-- Logs begin at Sat 2022-10-15 13:10:14 UTC, end at Sat 2022-10-15 14:11:28 UTC. -- Oct 15 13:34:19.982143 dv-cinder-861491-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 15 13:34:19.991929 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 15 13:34:19.993984 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Oct 15 13:34:19 2022] *** Oct 15 13:34:19.993984 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 15 13:34:19.993984 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 15 13:34:19.993984 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: nodename: dv-cinder-861491-3-iscsi Oct 15 13:34:19.993984 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: machine: x86_64 Oct 15 13:34:19.993984 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: clock source: unix Oct 15 13:34:19.993984 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: pcre jit disabled Oct 15 13:34:19.993984 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: detected number of CPU cores: 4 Oct 15 13:34:19.993984 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: current working directory: / Oct 15 13:34:19.993984 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: detected binary path: /usr/bin/uwsgi-core Oct 15 13:34:19.993984 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: your processes number limit is 47680 Oct 15 13:34:19.993984 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: your memory page size is 4096 bytes Oct 15 13:34:19.993984 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: detected max file descriptor number: 2048 Oct 15 13:34:19.993984 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: lock engine: pthread robust mutexes Oct 15 13:34:19.995287 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: thunder lock: enabled Oct 15 13:34:19.995287 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 15 13:34:19.995287 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 15 13:34:20.033189 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: Python main interpreter initialized at 0x5558d26ea9c0 Oct 15 13:34:20.033189 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: python threads support enabled Oct 15 13:34:20.033189 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: your server socket listen backlog is limited to 100 connections Oct 15 13:34:20.033189 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: your mercy for graceful operations on workers is 90 seconds Oct 15 13:34:20.033750 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: mapped 403077 bytes (393 KB) for 2 cores Oct 15 13:34:20.033750 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: *** Operational MODE: preforking *** Oct 15 13:34:20.033955 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: *** uWSGI is running in multiple interpreter mode *** Oct 15 13:34:20.033955 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: spawned uWSGI master process (pid: 87907) Oct 15 13:34:20.034161 dv-cinder-861491-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 15 13:34:20.034716 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: spawned uWSGI worker 1 (pid: 87915, cores: 1) Oct 15 13:34:20.035047 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: spawned uWSGI worker 2 (pid: 87916, cores: 1) Oct 15 13:34:20.035244 dv-cinder-861491-3-iscsi devstack@keystone.service[87907]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 15 13:34:24.425240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 13:34:24.425240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: from cryptography.utils import int_from_bytes Oct 15 13:34:24.426191 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 13:34:24.426191 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: from cryptography.utils import int_from_bytes Oct 15 13:34:24.476712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 15 13:34:24.476912 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 15 13:34:24.477090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] command line args: [] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 15 13:34:24.477354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 15 13:34:24.477493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ================================================================================ {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 15 13:34:24.477775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] admin_token = **** {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.478016 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] config_dir = [] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.478161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.478373 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] config_source = [] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.478503 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] debug = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.478739 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.479981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.479981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.479981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.479981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.479981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] list_limit = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.480303 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] log_config_append = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.480303 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.480582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] log_dir = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.480722 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] log_file = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.480881 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.481114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.481372 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.481616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.481806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.481961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.482215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.482458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.482687 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.482919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.483147 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.483381 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.483624 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.483867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.484160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.484653 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.484833 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] publish_errors = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.485048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.485281 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.485461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.485707 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.485941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.486178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] standard_threads = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.486410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.486980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.487165 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.487341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] use_journal = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.487521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] use_json = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.487682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] use_stderr = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.487856 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] use_syslog = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.488164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:24.488955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.489209 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.489477 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.489723 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.489976 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.490219 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.490471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.490713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.490963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.491211 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.491466 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.491703 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.491977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.492218 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.492468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.492716 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.492955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.493211 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.493469 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.493715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.493955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.494208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.494469 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.494721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.494969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.495205 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.495441 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.495675 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.495935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.496188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.496438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.496669 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.496902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.497149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.497398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.497639 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.497889 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.498134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.498361 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.498597 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.498829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.499060 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.499290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.499520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.499759 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.500109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.500344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.500584 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.500814 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.501053 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.501290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.501537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.501771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.502009 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.502247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.502483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.502718 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.502955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.503185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.503432 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.503703 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.504430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.504682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.504951 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.505196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.505432 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.505675 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.505915 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.506194 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.506434 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.506682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.506928 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.507169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.507407 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.507645 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.507917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.508161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.508399 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.508644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.508881 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.509130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.509368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.509605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.509842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.510078 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.510317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.510577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.510818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.511064 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.511304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.511544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.511785 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.512045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.512285 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.512521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.512751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.512981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.513211 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.513448 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.513679 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.513920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.514156 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.514404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.514641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.514893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.515152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.515390 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.515620 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] auth.external = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.515849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.516170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.516404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.516635 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] auth.password = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.516864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] auth.token = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.517102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.517338 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.517585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.517817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.518064 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.518311 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.518541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.518778 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] credential.caching = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.519027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.519346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.519690 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.519965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.520212 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.520450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.520699 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.520954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.521192 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.521443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.521733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.521980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.522242 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.522482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.522740 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.522976 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] federation.caching = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.523217 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.523464 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.523710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.523969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.524219 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.524455 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.524707 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.524946 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.525200 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.525437 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.525684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.525926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity.caching = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.526175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.526421 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.526658 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.527079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.527202 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.527357 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.527596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.527930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.528115 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.528347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.528579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.528813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.529075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.529307 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.529555 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.529813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.530064 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.530336 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.530577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.530816 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.531048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.531286 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.531516 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.531755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.532020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.532264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.532512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.532746 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.532992 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.533240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.533478 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.533725 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.533970 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.534201 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.534442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.534674 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.535120 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.535369 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.535622 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.535862 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.536268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.536526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.536778 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.537014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.537248 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.537501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.537750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.537991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.538229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.538470 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.538704 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.538947 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.539208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.539443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.539692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.539965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.540214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.540450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.540683 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.540921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.541158 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.541396 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.541628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.541874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.542122 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.542370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.542616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.542860 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.543091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.543339 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.543585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.543816 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.544079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.544315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.544561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.544810 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.545048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.545303 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.545535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.545782 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.546027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.546266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.546504 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.546752 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.546996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.547230 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.547468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.547714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] resource.caching = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.547997 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.548248 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.548487 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.548738 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.548990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.549228 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.549476 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.549715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.549955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.550192 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] role.caching = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.550444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] role.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.550993 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.551247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.551500 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.551750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.552027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.552277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.552526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.552775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.553024 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.553260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.553510 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.553758 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.554005 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.554252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.554680 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.554920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.555169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.555420 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.555666 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.555941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.556184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.556440 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.556691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.556927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.557160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.557395 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.557627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.557880 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.558134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.558381 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.558621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.558856 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.559086 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.559323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] token.caching = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.559562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.559812 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.560075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.560332 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.560584 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.560823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.561070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.561316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.561565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.561803 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.562044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.562285 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.562531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.562806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.563038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.563283 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.563548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.563809 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.connection = **** {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.564268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.564521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.564760 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.565005 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.565242 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.565481 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.565716 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.565954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.566230 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.566464 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.566709 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.566948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.567195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.567433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.567679 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.567928 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.568172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.568407 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:24.568580 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 15 13:34:24.598403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.598587 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.598853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.607340 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.636166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.636440 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.690230 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.739344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.739543 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.897220 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.897461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.897660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.897911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.898131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.898327 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.921849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.922098 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.922325 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.922545 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.922773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.922984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.923184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.934532 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.947238 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.947368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.947622 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.947862 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.948072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.948321 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.948508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:24.965896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.966067 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 13:34:24.966441 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.966622 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 13:34:24.966927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.967191 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.967457 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.967717 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.977653 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.977934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.978209 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.978494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.985452 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:24.988747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:24.989054 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.989332 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.989621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.989912 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.990177 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.990454 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.990729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.990992 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.991268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:24.991547 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.014334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.015793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.016108 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.020839 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.021126 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.021416 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.027721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.028049 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.033813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.034120 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.034405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.034688 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.034997 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.035421 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.035699 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.036023 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.051777 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.052106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.056072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.059275 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.059576 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.063707 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.067426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.070627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.070931 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.071196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.071479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.071761 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.072069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.087751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.088068 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.088337 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.088604 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.095399 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.096964 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.098625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.098901 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.101620 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.101912 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.102200 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.102485 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.102884 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.114355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.114646 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.114927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.115214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.115602 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.115911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.116193 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.131529 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.134470 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.137358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.139030 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.140610 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.140961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.141424 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.148031 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.149499 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.152389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.152786 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.153078 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.153361 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.161322 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.161615 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.161912 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.168779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.169070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.169350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.169631 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.169906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.170198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.170475 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.170752 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.171023 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.171306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.171577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.171857 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.172227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.172497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.206612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5558d26ea9c0 pid: 87915 (default app) Oct 15 13:34:25.210019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c183c-f379-46e4-802f-9840d11b2641 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:25.210312 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c183c-f379-46e4-802f-9840d11b2641 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:25.210580 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c183c-f379-46e4-802f-9840d11b2641 None None] PATH_INFO: `/v3` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:25.214771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Sat Oct 15 13:34:25 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:25.451737 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 13:34:25.451737 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: from cryptography.utils import int_from_bytes Oct 15 13:34:25.452438 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 15 13:34:25.452438 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: from cryptography.utils import int_from_bytes Oct 15 13:34:25.498589 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 15 13:34:25.498762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 15 13:34:25.498874 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] command line args: [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 15 13:34:25.499163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 15 13:34:25.499305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ================================================================================ {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 15 13:34:25.499567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] admin_token = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.499800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] config_dir = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.500065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.500189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] config_source = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.500358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] debug = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.500523 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.500792 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.500915 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.501195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.501429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.501655 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] list_limit = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.501778 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] log_config_append = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.501985 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.502109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] log_dir = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.502299 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] log_file = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.502527 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.502764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.502998 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.503234 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.503407 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.503600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.503908 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.504125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.504353 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.504579 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.504805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.505032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.505275 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.505515 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.505781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.506254 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.506429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] publish_errors = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.506653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.506874 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.507046 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.507282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.507508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.507735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] standard_threads = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.507984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.508219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.508390 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.508561 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] use_journal = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.508734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] use_json = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.508907 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] use_stderr = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.509077 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] use_syslog = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.509300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 15 13:34:25.509550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.509784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.510035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.510269 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.510511 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.510749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.510989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.511219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.511458 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.511697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.511963 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.512193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.512430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.512661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.512899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.513138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.513369 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.513615 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.513861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.514099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.514326 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.514572 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.514824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.515067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.515307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.515531 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.515756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.516003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.516232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.516471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.516710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.516933 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.517156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.517394 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.517633 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.517865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.518104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.518329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.518561 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.518787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.519010 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.519256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.519479 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.519832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.520095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.520340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.520567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.520802 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.521028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.521259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.521489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.521728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.521951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.522181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.522410 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.522637 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.522865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.523093 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.523325 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.523568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.523799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.524052 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.524278 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.524529 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.524757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.524987 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.525208 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.525440 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.525698 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.525939 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.526183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.526423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.526655 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.526886 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.527116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.527357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.527589 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.527818 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.528081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.528314 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.528566 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.528797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.529029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.529258 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.529487 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.529716 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.529974 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.530208 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.530448 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.530687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.530920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.531156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.531388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.531619 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.531849 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.532097 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.532321 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.532548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.532779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.533002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.533237 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.533466 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.533707 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.533938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.534187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.534439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.534674 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.534986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] auth.external = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.535395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.535764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.536016 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.536246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] auth.password = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.536473 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] auth.token = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.536709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.536943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.537183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.537411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.537651 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.537895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.538122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.538353 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] credential.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.538593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.538836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.539078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.539330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.539568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.539797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.540061 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.540310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.540541 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.540786 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.541081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.541323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.541582 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.541819 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.542071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.542302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.542536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.542787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.543031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.543260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.543504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.543737 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.544008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.544242 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.544491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.544722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.544963 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.545195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.545482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.545678 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.545910 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.546140 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.546382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.546608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.546839 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.547083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.547328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.547556 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.547780 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.548029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.548271 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.548510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.548751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.549000 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.549242 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.549535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.549738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.549971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.550198 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.550451 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.550685 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.550947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.551183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.551416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.551656 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.551890 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.552144 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.552385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.552617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.552856 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.553094 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.553317 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.553551 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.553775 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.554005 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.554234 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.554473 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.554704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.554936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.555178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.555432 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.555661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.555904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.556392 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.556637 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.556869 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.557103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.557334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.557561 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.557796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.558051 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.558279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.558522 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.558766 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.559008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.559238 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.559462 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.559693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.559954 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.560187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.560414 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.560655 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.560901 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.561142 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.561382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.561622 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.561846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.562091 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.562324 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.562556 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.562798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.563031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.563272 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.563515 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.563747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.564018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.564245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.564487 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.564719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.564950 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.565183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.565423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.565660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.565884 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.566192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.566420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.566664 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.566905 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.567139 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.567382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.567627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.567858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.568119 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.568350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.568582 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.568812 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] role.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.569051 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] role.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.569277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.569520 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.569761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.570004 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.570244 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.570483 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.570722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.570968 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.571210 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.571440 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.571692 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.571955 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.572198 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.572439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.572860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.573090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.573334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.573578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.573820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.574062 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.574293 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.574542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.574787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.575021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.575248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.575475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.575702 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.575969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.576220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.576462 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.576697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.576929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.577156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.577387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.577619 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.577860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.578091 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.578340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.578581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.578812 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.579050 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.579287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.579527 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.579757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.580012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.580243 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.580482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.580760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.580986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.581224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.581483 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.581829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.connection = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.582036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.582276 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.582508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.582744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.582974 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.583207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.583435 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.583665 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.583922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.584154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.584394 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.584626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.584868 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.585099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.585352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.585578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.585817 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.586048 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 15 13:34:25.586215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 15 13:34:25.612821 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.613037 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.613291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.621624 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.639907 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.640183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.677688 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.712340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.712596 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.754214 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.754398 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.754646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.754864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.755081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.755292 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.769837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.770121 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.770351 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.770568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.770795 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.771001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.771194 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.871796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.874966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.875192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.875411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.875598 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.875840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.876151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.876360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 15 13:34:25.886400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.886672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 13:34:25.886937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.887128 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 15 13:34:25.887398 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.887631 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.887922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.888202 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.897919 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.898198 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.898476 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.898755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.905737 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.908635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.908916 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.909195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.909468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.909743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.910086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.910359 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.910638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.910905 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.911169 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.911440 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.933773 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.935222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.935504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.940027 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.940317 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.940595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.947010 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.947293 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.951204 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.951494 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.951775 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.952089 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.952354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.952750 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.953021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.953276 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.968941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.969257 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.973287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.976583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.976892 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.981337 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.984523 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:25.987919 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.988310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.988591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.988991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.989278 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:25.989567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.006672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:26.006897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.007168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.007426 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.014030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.015653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.017179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.017452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.020148 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:26.020439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.020715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.021005 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.021289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.032459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:26.032745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.033036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.033317 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.033595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.033866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.034150 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.049260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:26.052338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:26.055164 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.056598 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.058039 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:26.058319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.058606 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.065048 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.066499 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:26.069395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.069652 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.069963 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.070232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.077998 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:26.078261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.078558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.085570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 15 13:34:26.085854 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.086131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.086413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.086683 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.086963 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.087233 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.087508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.087785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.088072 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.088333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.088598 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.088868 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.089125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 15 13:34:26.121911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5558d26ea9c0 pid: 87916 (default app) Oct 15 13:34:32.183089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa742258-8a76-428a-af2a-f920748f60d7 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:32.183635 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa742258-8a76-428a-af2a-f920748f60d7 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:32.183635 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa742258-8a76-428a-af2a-f920748f60d7 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:32.185128 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 2/2] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:32.194623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c700181-68d2-4462-960e-f65fa3b83408 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:32.195356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c700181-68d2-4462-960e-f65fa3b83408 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:32.195356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c700181-68d2-4462-960e-f65fa3b83408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:32.269652 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: INFO dbcounter [None req-3c700181-68d2-4462-960e-f65fa3b83408 None None] Registered counter for database keystone Oct 15 13:34:32.290522 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG oslo_db.sqlalchemy.engines [None req-3c700181-68d2-4462-960e-f65fa3b83408 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87916) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 13:34:32.301249 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writer thread running {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 13:34:32.589341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG passlib.handlers.bcrypt [None req-3c700181-68d2-4462-960e-f65fa3b83408 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87916) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 15 13:34:32.589782 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG passlib.handlers.bcrypt [None req-3c700181-68d2-4462-960e-f65fa3b83408 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87916) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 15 13:34:32.650547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-3c700181-68d2-4462-960e-f65fa3b83408 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:32.652202 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3c700181-68d2-4462-960e-f65fa3b83408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:32.791102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1/3] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 601 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 15 13:34:33.088685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa742258-8a76-428a-af2a-f920748f60d7 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:33.160404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: INFO dbcounter [None req-aa742258-8a76-428a-af2a-f920748f60d7 None None] Registered counter for database keystone Oct 15 13:34:33.183448 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG oslo_db.sqlalchemy.engines [None req-aa742258-8a76-428a-af2a-f920748f60d7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87915) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 15 13:34:33.190614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writer thread running {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 15 13:34:33.312964 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa742258-8a76-428a-af2a-f920748f60d7 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:33.318100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37926de-f6c3-43da-be53-336a5037eecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:33.318799 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37926de-f6c3-43da-be53-336a5037eecf admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:33.319118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37926de-f6c3-43da-be53-336a5037eecf admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:33.319433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37926de-f6c3-43da-be53-336a5037eecf admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:33.328820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37926de-f6c3-43da-be53-336a5037eecf admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:33.381212 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b37926de-f6c3-43da-be53-336a5037eecf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 15 13:34:33.381585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37926de-f6c3-43da-be53-336a5037eecf admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:33.392198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 3/4] 10.222.0.36 () {58 vars in 1324 bytes} [Sat Oct 15 13:34:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 13:34:33.399149 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c700181-68d2-4462-960e-f65fa3b83408 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:33.415487 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c700181-68d2-4462-960e-f65fa3b83408 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:33.421927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21b324-6483-4e4f-bf58-22cdffbc6dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:33.423941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21b324-6483-4e4f-bf58-22cdffbc6dfa admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:33.424436 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21b324-6483-4e4f-bf58-22cdffbc6dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:33.424894 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21b324-6483-4e4f-bf58-22cdffbc6dfa admin admin] PATH_INFO: `/v3/projects/443104cd704b44c6867e1bc065e90d39` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:33.427782 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e21b324-6483-4e4f-bf58-22cdffbc6dfa admin admin] RBAC: Authorizing `identity:get_project(project_id=443104cd704b44c6867e1bc065e90d39)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:33.533158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7e21b324-6483-4e4f-bf58-22cdffbc6dfa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 15 13:34:33.533731 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e21b324-6483-4e4f-bf58-22cdffbc6dfa admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:33.543040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 2/5] 10.222.0.36 () {56 vars in 1207 bytes} [Sat Oct 15 13:34:33 2022] GET /identity/v3/projects/443104cd704b44c6867e1bc065e90d39 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:35.884855 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb10cc-eebd-4687-b845-a5b04d508582 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:35.885818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb10cc-eebd-4687-b845-a5b04d508582 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:35.885818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb10cc-eebd-4687-b845-a5b04d508582 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:35.892096 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 4/6] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:35.898990 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491e912-66c9-4517-9d78-334f919069f3 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:35.899580 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491e912-66c9-4517-9d78-334f919069f3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:35.899580 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491e912-66c9-4517-9d78-334f919069f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:35.921513 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6491e912-66c9-4517-9d78-334f919069f3 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:35.923066 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6491e912-66c9-4517-9d78-334f919069f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:35.932640 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 3/7] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 15 13:34:36.203784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb10cc-eebd-4687-b845-a5b04d508582 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:36.225720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb10cc-eebd-4687-b845-a5b04d508582 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:36.233443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71338e-97ff-4c95-8eae-0993ed06058b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:36.234490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71338e-97ff-4c95-8eae-0993ed06058b admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:36.234972 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71338e-97ff-4c95-8eae-0993ed06058b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:36.235432 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71338e-97ff-4c95-8eae-0993ed06058b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:36.250958 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71338e-97ff-4c95-8eae-0993ed06058b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:36.254477 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71338e-97ff-4c95-8eae-0993ed06058b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:36.271944 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 5/8] 10.222.0.36 () {58 vars in 1324 bytes} [Sat Oct 15 13:34:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:34:36.278446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491e912-66c9-4517-9d78-334f919069f3 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:36.289231 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491e912-66c9-4517-9d78-334f919069f3 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:36.294993 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2818bc2-be6a-4b96-8ab5-b7eca90fe81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:36.295668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2818bc2-be6a-4b96-8ab5-b7eca90fe81b admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:36.295983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2818bc2-be6a-4b96-8ab5-b7eca90fe81b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:36.296253 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2818bc2-be6a-4b96-8ab5-b7eca90fe81b admin admin] PATH_INFO: `/v3/users/d6c4e60be4154870948fd9be3f7e4091` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:36.297805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2818bc2-be6a-4b96-8ab5-b7eca90fe81b admin admin] RBAC: Authorizing `identity:get_user(user_id=d6c4e60be4154870948fd9be3f7e4091)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:36.299746 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2818bc2-be6a-4b96-8ab5-b7eca90fe81b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:36.302069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 4/9] 10.222.0.36 () {56 vars in 1198 bytes} [Sat Oct 15 13:34:36 2022] GET /identity/v3/users/d6c4e60be4154870948fd9be3f7e4091 => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:38.671629 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc272f66-07e4-4ef1-9a86-4f1ba8e88303 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:38.672262 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc272f66-07e4-4ef1-9a86-4f1ba8e88303 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:38.672615 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc272f66-07e4-4ef1-9a86-4f1ba8e88303 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:38.675313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 6/10] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:38.681572 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9293efc-19d1-4723-a7de-e288e063f7c3 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:38.682352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9293efc-19d1-4723-a7de-e288e063f7c3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:38.682508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9293efc-19d1-4723-a7de-e288e063f7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:38.699991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a098fd-f7dd-4bf1-b6b4-4b61f6bfd048 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:38.700258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a098fd-f7dd-4bf1-b6b4-4b61f6bfd048 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:38.700518 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a098fd-f7dd-4bf1-b6b4-4b61f6bfd048 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:38.701872 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 7/11] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:34:38.702884 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b9293efc-19d1-4723-a7de-e288e063f7c3 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:38.704135 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b9293efc-19d1-4723-a7de-e288e063f7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:38.710344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421c157-af78-4bef-a9be-0307b08ac181 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:38.710695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421c157-af78-4bef-a9be-0307b08ac181 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:38.711036 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421c157-af78-4bef-a9be-0307b08ac181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:38.759794 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG passlib.handlers.bcrypt [None req-1421c157-af78-4bef-a9be-0307b08ac181 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87915) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 15 13:34:38.760383 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG passlib.handlers.bcrypt [None req-1421c157-af78-4bef-a9be-0307b08ac181 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87915) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 15 13:34:38.798316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 5/12] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:34:38.811136 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-1421c157-af78-4bef-a9be-0307b08ac181 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:38.812814 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1421c157-af78-4bef-a9be-0307b08ac181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:38.929505 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 8/13] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:34:39.196380 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9293efc-19d1-4723-a7de-e288e063f7c3 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:39.234231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421c157-af78-4bef-a9be-0307b08ac181 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:39.237613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:39.337598 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9293efc-19d1-4723-a7de-e288e063f7c3 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:39.383892 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492052ed-2b56-41ca-b83e-875d5ab4544f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:39.384537 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492052ed-2b56-41ca-b83e-875d5ab4544f None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:39.384808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492052ed-2b56-41ca-b83e-875d5ab4544f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:39.385067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492052ed-2b56-41ca-b83e-875d5ab4544f None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:39.385746 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492052ed-2b56-41ca-b83e-875d5ab4544f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:39.387173 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492052ed-2b56-41ca-b83e-875d5ab4544f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:39.389365 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421c157-af78-4bef-a9be-0307b08ac181 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:39.394904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:39.398974 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.sql.core [None req-492052ed-2b56-41ca-b83e-875d5ab4544f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 13:34:39.398974 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 13:34:39.398974 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [parameters: {'id': 'bcec2f9399fc4fcdaa6458a364e61a68', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 13:34:39.398974 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87916) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 13:34:39.420564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-492052ed-2b56-41ca-b83e-875d5ab4544f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 13:34:39.420564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:34:39.420564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 13:34:39.420564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 13:34:39.420564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 13:34:39.420564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 13:34:39.420564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 13:34:39.420564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 13:34:39.420564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 13:34:39.420564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application conn.query(q) Oct 15 13:34:39.420564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 13:34:39.420564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 13:34:39.420564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 13:34:39.420564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result.read() Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 15 13:34:39.422175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application next(self.gen) Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application yield resource Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application next(self.gen) Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise self.value Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application session.commit() Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 13:34:39.423898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.session.flush() Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application compat.raise_( Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 13:34:39.425587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 13:34:39.427177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application conn.query(q) Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result.read() Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 13:34:39.428751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [parameters: {'id': 'bcec2f9399fc4fcdaa6458a364e61a68', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 15 13:34:39.430229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:34:39.431535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:34:39.435542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 15 13:34:39.435542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 15 13:34:39.435542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:34:39.435542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:34:39.435542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 15 13:34:39.435542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 15 13:34:39.435542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:34:39.435542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:34:39.435542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 15 13:34:39.435542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 15 13:34:39.435542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 13:34:39.435542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:34:39.435542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 6/14] 10.222.0.36 () {60 vars in 1137 bytes} [Sat Oct 15 13:34:39 2022] POST /identity/v3/domains => generated 190 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:34:39.435542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492052ed-2b56-41ca-b83e-875d5ab4544f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:39.443515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffa3b6-8b7b-4bc3-bdc2-7b09bc2a8b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:39.445174 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffa3b6-8b7b-4bc3-bdc2-7b09bc2a8b02 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:39.445455 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffa3b6-8b7b-4bc3-bdc2-7b09bc2a8b02 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:39.445718 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffa3b6-8b7b-4bc3-bdc2-7b09bc2a8b02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:39.463529 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bffa3b6-8b7b-4bc3-bdc2-7b09bc2a8b02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-5bffa3b6-8b7b-4bc3-bdc2-7b09bc2a8b02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:34:39.476131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:34:39.477898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:34:39.479442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:34:39.479442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:34:39.479442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:34:39.479442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:34:39.479442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:34:39.479442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:34:39.479442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:34:39.479442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:34:39.479442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:34:39.479442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:34:39.479442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:34:39.479442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:34:39.479442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 9/15] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:34:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:34:39.485426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffa3b6-8b7b-4bc3-bdc2-7b09bc2a8b02 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:39.553915 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:39.584761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492052ed-2b56-41ca-b83e-875d5ab4544f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:39.615179 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffa3b6-8b7b-4bc3-bdc2-7b09bc2a8b02 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:39.633813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c9849-5232-4b64-a55b-2d227a5cc9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:39.634473 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c9849-5232-4b64-a55b-2d227a5cc9fa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:39.634747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c9849-5232-4b64-a55b-2d227a5cc9fa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:39.635011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c9849-5232-4b64-a55b-2d227a5cc9fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:39.647723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c9849-5232-4b64-a55b-2d227a5cc9fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:39.649527 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c9849-5232-4b64-a55b-2d227a5cc9fa None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-359c9849-5232-4b64-a55b-2d227a5cc9fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:34:39.666588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:34:39.668358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:34:39.669808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:34:39.669808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:34:39.669808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:34:39.669808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 7/16] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:34:39.674032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c9849-5232-4b64-a55b-2d227a5cc9fa None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:39.677282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d10ab-9b04-4998-ac90-bba0fac65e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:39.678274 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d10ab-9b04-4998-ac90-bba0fac65e6a None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:39.678688 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d10ab-9b04-4998-ac90-bba0fac65e6a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:39.679072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d10ab-9b04-4998-ac90-bba0fac65e6a None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:39.679903 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d10ab-9b04-4998-ac90-bba0fac65e6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:34:39.680325 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d10ab-9b04-4998-ac90-bba0fac65e6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:39.681953 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d10ab-9b04-4998-ac90-bba0fac65e6a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:39.696716 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 10/17] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:34:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:34:39.706992 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d10ab-9b04-4998-ac90-bba0fac65e6a None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:39.739522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:39.822783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c9849-5232-4b64-a55b-2d227a5cc9fa None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:39.833933 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d10ab-9b04-4998-ac90-bba0fac65e6a None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:39.868562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fff0a-e8a7-4e7a-9a0f-7ae71109478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:39.869531 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fff0a-e8a7-4e7a-9a0f-7ae71109478d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:39.872861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fff0a-e8a7-4e7a-9a0f-7ae71109478d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:39.872861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fff0a-e8a7-4e7a-9a0f-7ae71109478d None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:39.872861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fff0a-e8a7-4e7a-9a0f-7ae71109478d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:34:39.872861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fff0a-e8a7-4e7a-9a0f-7ae71109478d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:39.874071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fff0a-e8a7-4e7a-9a0f-7ae71109478d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:39.882254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea7b44-d082-4c3a-b28c-f44f0b70bfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:39.882993 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea7b44-d082-4c3a-b28c-f44f0b70bfd6 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:39.883332 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea7b44-d082-4c3a-b28c-f44f0b70bfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:39.883654 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea7b44-d082-4c3a-b28c-f44f0b70bfd6 None admin] PATH_INFO: `/v3/users/d6c4e60be4154870948fd9be3f7e4091` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:39.885790 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea7b44-d082-4c3a-b28c-f44f0b70bfd6 None admin] RBAC: Authorizing `identity:get_user(user_id=d6c4e60be4154870948fd9be3f7e4091)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:39.888188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea7b44-d082-4c3a-b28c-f44f0b70bfd6 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:39.888834 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:39.891643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 11/18] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:34:39 2022] GET /identity/v3/users/d6c4e60be4154870948fd9be3f7e4091 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:39.897047 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea7b44-d082-4c3a-b28c-f44f0b70bfd6 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:39.899782 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 8/19] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:40.048000 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:40.060048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea7b44-d082-4c3a-b28c-f44f0b70bfd6 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:40.118871 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cf0d2-e813-4bec-8048-de326a88ad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:40.119917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf0d2-e813-4bec-8048-de326a88ad99 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:40.120359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf0d2-e813-4bec-8048-de326a88ad99 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:40.120769 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf0d2-e813-4bec-8048-de326a88ad99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:40.122835 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cf0d2-e813-4bec-8048-de326a88ad99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:40.125384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cf0d2-e813-4bec-8048-de326a88ad99 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:40.128349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 12/20] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:34:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:34:40.136874 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fff0a-e8a7-4e7a-9a0f-7ae71109478d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:40.266289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fff0a-e8a7-4e7a-9a0f-7ae71109478d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:40.306383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1454c-b0c6-446e-8b47-a60ac6266340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:40.307000 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1454c-b0c6-446e-8b47-a60ac6266340 None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:40.307720 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1454c-b0c6-446e-8b47-a60ac6266340 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:40.307720 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1454c-b0c6-446e-8b47-a60ac6266340 None admin] PATH_INFO: `/v3/domains/default/users/d6c4e60be4154870948fd9be3f7e4091/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:40.309537 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1454c-b0c6-446e-8b47-a60ac6266340 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d6c4e60be4154870948fd9be3f7e4091, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:40.311072 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1454c-b0c6-446e-8b47-a60ac6266340 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:40.429755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 9/21] 10.222.0.36 () {58 vars in 1361 bytes} [Sat Oct 15 13:34:40 2022] PUT /identity/v3/domains/default/users/d6c4e60be4154870948fd9be3f7e4091/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:34:42.945620 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f7d9-ff79-4d86-9ef3-a76b69e1dc34 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:42.946964 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f7d9-ff79-4d86-9ef3-a76b69e1dc34 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:42.947669 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f7d9-ff79-4d86-9ef3-a76b69e1dc34 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:42.950200 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 13/22] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:42.958098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823f079-e744-4a95-bbc5-ecafce87af86 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:42.959415 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823f079-e744-4a95-bbc5-ecafce87af86 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:42.960144 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823f079-e744-4a95-bbc5-ecafce87af86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:43.001044 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b823f079-e744-4a95-bbc5-ecafce87af86 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:43.004191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b823f079-e744-4a95-bbc5-ecafce87af86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:43.130395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:43.165966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 10/23] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:34:43.425762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f01d-4f94-43fd-8f83-e8e3b6809dc6 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:43.426410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f01d-4f94-43fd-8f83-e8e3b6809dc6 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:43.426902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f01d-4f94-43fd-8f83-e8e3b6809dc6 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:43.428765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 14/24] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:43.436081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaac7d-f564-4e52-bf55-d77056532d23 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:43.436674 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaac7d-f564-4e52-bf55-d77056532d23 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:43.437172 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaac7d-f564-4e52-bf55-d77056532d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:43.458484 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-1eeaac7d-f564-4e52-bf55-d77056532d23 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:43.460009 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1eeaac7d-f564-4e52-bf55-d77056532d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:43.561531 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 11/25] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:34:43.673244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f01d-4f94-43fd-8f83-e8e3b6809dc6 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:43.796101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f9425-6eac-4545-b62a-da2ca9ed78b6 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:43.796282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f9425-6eac-4545-b62a-da2ca9ed78b6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:43.796568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f9425-6eac-4545-b62a-da2ca9ed78b6 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:43.800894 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 12/26] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:34:43.807941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d7a84-fe4e-43fa-a6f3-52fdbc18e0d1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:43.808228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d7a84-fe4e-43fa-a6f3-52fdbc18e0d1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:43.808502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d7a84-fe4e-43fa-a6f3-52fdbc18e0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:43.811509 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f01d-4f94-43fd-8f83-e8e3b6809dc6 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:43.847369 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-598d7a84-fe4e-43fa-a6f3-52fdbc18e0d1 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:43.850185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-598d7a84-fe4e-43fa-a6f3-52fdbc18e0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:43.911407 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2fec0-614d-4e6a-802e-7efb393ee977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:43.912447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2fec0-614d-4e6a-802e-7efb393ee977 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:43.913579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2fec0-614d-4e6a-802e-7efb393ee977 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:43.914235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2fec0-614d-4e6a-802e-7efb393ee977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:43.959756 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa2fec0-614d-4e6a-802e-7efb393ee977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:43.966564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa2fec0-614d-4e6a-802e-7efb393ee977 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-9fa2fec0-614d-4e6a-802e-7efb393ee977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:34:43.999223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:34:44.000175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:34:44.000948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:34:44.000948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:34:44.000948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:34:44.009391 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 15/27] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:34:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:34:44.017553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2fec0-614d-4e6a-802e-7efb393ee977 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:44.041853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 13/28] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:34:44.053273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:44.154145 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2fec0-614d-4e6a-802e-7efb393ee977 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:44.194060 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a339367-f6e1-42e3-8c12-f916e6c4a330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:44.195073 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a339367-f6e1-42e3-8c12-f916e6c4a330 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:44.195544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a339367-f6e1-42e3-8c12-f916e6c4a330 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:44.196043 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a339367-f6e1-42e3-8c12-f916e6c4a330 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:44.196889 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a339367-f6e1-42e3-8c12-f916e6c4a330 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:34:44.197373 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a339367-f6e1-42e3-8c12-f916e6c4a330 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:44.198991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a339367-f6e1-42e3-8c12-f916e6c4a330 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:44.222943 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 16/29] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:44.231320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d7a84-fe4e-43fa-a6f3-52fdbc18e0d1 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:44.242400 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a339367-f6e1-42e3-8c12-f916e6c4a330 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:44.342671 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:44.375942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:44.433539 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a339367-f6e1-42e3-8c12-f916e6c4a330 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:44.482205 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233f220-def3-408b-85f5-6baa117c7b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:44.483221 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233f220-def3-408b-85f5-6baa117c7b56 None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:44.483779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233f220-def3-408b-85f5-6baa117c7b56 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:44.484273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233f220-def3-408b-85f5-6baa117c7b56 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:44.485221 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233f220-def3-408b-85f5-6baa117c7b56 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:44.487274 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233f220-def3-408b-85f5-6baa117c7b56 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:44.499690 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d7a84-fe4e-43fa-a6f3-52fdbc18e0d1 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:44.630564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285dffc-2f9a-417a-a172-6d70c08cef91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:44.632031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285dffc-2f9a-417a-a172-6d70c08cef91 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:44.632699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285dffc-2f9a-417a-a172-6d70c08cef91 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:44.633327 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285dffc-2f9a-417a-a172-6d70c08cef91 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:44.634646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285dffc-2f9a-417a-a172-6d70c08cef91 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:44.637415 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285dffc-2f9a-417a-a172-6d70c08cef91 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:44.661076 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 17/30] 10.222.0.36 () {60 vars in 1131 bytes} [Sat Oct 15 13:34:44 2022] POST /identity/v3/roles => generated 219 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:34:44.662960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56e42-1d07-43ce-8536-b656de5fd0dd None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:44.663453 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56e42-1d07-43ce-8536-b656de5fd0dd None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:44.664054 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56e42-1d07-43ce-8536-b656de5fd0dd None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:44.665606 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 18/31] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:44.674004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bf88c-400d-4dbd-8599-fdec87420760 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:44.674662 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bf88c-400d-4dbd-8599-fdec87420760 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:44.675187 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bf88c-400d-4dbd-8599-fdec87420760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:44.712219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 14/32] 10.222.0.36 () {60 vars in 1141 bytes} [Sat Oct 15 13:34:44 2022] POST /identity/v3/projects => generated 301 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:34:44.716846 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-7b7bf88c-400d-4dbd-8599-fdec87420760 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:44.718695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7b7bf88c-400d-4dbd-8599-fdec87420760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:44.737980 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285dffc-2f9a-417a-a172-6d70c08cef91 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:44.811210 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 19/33] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:34:44.906034 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:44.910138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285dffc-2f9a-417a-a172-6d70c08cef91 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:44.961258 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a0f8b-fdc4-4027-8dd8-33c94322443a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:44.962306 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a0f8b-fdc4-4027-8dd8-33c94322443a None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:44.962802 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a0f8b-fdc4-4027-8dd8-33c94322443a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:44.963361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a0f8b-fdc4-4027-8dd8-33c94322443a None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:44.964382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a0f8b-fdc4-4027-8dd8-33c94322443a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:44.966091 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a0f8b-fdc4-4027-8dd8-33c94322443a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:45.001687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 15/34] 10.222.0.36 () {60 vars in 1131 bytes} [Sat Oct 15 13:34:44 2022] POST /identity/v3/roles => generated 225 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:34:45.154246 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7bf88c-400d-4dbd-8599-fdec87420760 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:45.239469 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:45.293323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7bf88c-400d-4dbd-8599-fdec87420760 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:45.348856 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd7e26-8d8d-41b3-bd85-85cfd69a893b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:45.349646 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd7e26-8d8d-41b3-bd85-85cfd69a893b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:45.350042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd7e26-8d8d-41b3-bd85-85cfd69a893b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:45.350375 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd7e26-8d8d-41b3-bd85-85cfd69a893b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:45.368072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd7e26-8d8d-41b3-bd85-85cfd69a893b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-96fd7e26-8d8d-41b3-bd85-85cfd69a893b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:34:45.378214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:34:45.379152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:34:45.380044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:34:45.380044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:34:45.380044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:34:45.380044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:34:45.380044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:34:45.380044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:34:45.380044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:34:45.380044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:34:45.380044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:34:45.380044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:34:45.380044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:34:45.380044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:34:45.380636 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 20/35] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:34:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:34:45.389925 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a0f8b-fdc4-4027-8dd8-33c94322443a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:45.419001 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de6200-a8e1-4200-a070-af6393040969 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:45.419351 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de6200-a8e1-4200-a070-af6393040969 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:45.419842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de6200-a8e1-4200-a070-af6393040969 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:45.422584 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 21/36] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:45.429900 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326913-f108-4648-8087-95521d94331d None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:45.430397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326913-f108-4648-8087-95521d94331d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:45.430846 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326913-f108-4648-8087-95521d94331d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:45.454866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-c6326913-f108-4648-8087-95521d94331d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:45.456390 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c6326913-f108-4648-8087-95521d94331d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:45.520544 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a0f8b-fdc4-4027-8dd8-33c94322443a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:45.539205 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 22/37] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:34:45.541286 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfb50c-a485-4c32-ba2f-54cd45540a1d None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:45.541712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfb50c-a485-4c32-ba2f-54cd45540a1d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:45.542131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfb50c-a485-4c32-ba2f-54cd45540a1d None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:45.544344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 23/38] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:45.558004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b98466-ae94-4381-b9a3-36ee949045c6 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:45.558289 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b98466-ae94-4381-b9a3-36ee949045c6 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:45.558568 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b98466-ae94-4381-b9a3-36ee949045c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:45.561980 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e9bc7-74ca-4385-9e64-3da8eedcf633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:45.562626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e9bc7-74ca-4385-9e64-3da8eedcf633 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:45.562911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e9bc7-74ca-4385-9e64-3da8eedcf633 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:45.563180 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e9bc7-74ca-4385-9e64-3da8eedcf633 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:45.563868 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e9bc7-74ca-4385-9e64-3da8eedcf633 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:34:45.564219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e9bc7-74ca-4385-9e64-3da8eedcf633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:45.565861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e9bc7-74ca-4385-9e64-3da8eedcf633 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:45.578952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-e4b98466-ae94-4381-b9a3-36ee949045c6 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:45.579536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 16/39] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:34:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:45.580853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e4b98466-ae94-4381-b9a3-36ee949045c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:45.586173 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e9bc7-74ca-4385-9e64-3da8eedcf633 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:45.615761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:45.644362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:45.674196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 24/40] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:34:45.721799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e9bc7-74ca-4385-9e64-3da8eedcf633 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:45.770240 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cee8ff-79a2-4165-845b-ed7cc37c1294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:45.770897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee8ff-79a2-4165-845b-ed7cc37c1294 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:45.771170 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee8ff-79a2-4165-845b-ed7cc37c1294 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:45.771434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee8ff-79a2-4165-845b-ed7cc37c1294 None admin] PATH_INFO: `/v3/users/d6c4e60be4154870948fd9be3f7e4091` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:45.772636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cee8ff-79a2-4165-845b-ed7cc37c1294 None admin] RBAC: Authorizing `identity:get_user(user_id=d6c4e60be4154870948fd9be3f7e4091)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:45.774442 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cee8ff-79a2-4165-845b-ed7cc37c1294 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:45.776354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 17/41] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:34:45 2022] GET /identity/v3/users/d6c4e60be4154870948fd9be3f7e4091 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:34:45.782566 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b98466-ae94-4381-b9a3-36ee949045c6 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:45.817409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cee8ff-79a2-4165-845b-ed7cc37c1294 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:45.908483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b98466-ae94-4381-b9a3-36ee949045c6 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:45.948084 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f2c21-2c50-40ab-94f9-9814b8211e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:45.948682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f2c21-2c50-40ab-94f9-9814b8211e89 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:45.948964 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f2c21-2c50-40ab-94f9-9814b8211e89 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:45.949235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f2c21-2c50-40ab-94f9-9814b8211e89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:45.950427 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f2c21-2c50-40ab-94f9-9814b8211e89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:45.952090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f2c21-2c50-40ab-94f9-9814b8211e89 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:45.953737 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 25/42] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:34:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:45.954521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f2c21-2c50-40ab-94f9-9814b8211e89 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:45.960898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:45.972310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cee8ff-79a2-4165-845b-ed7cc37c1294 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:46.014608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:46.017504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764cea2-9522-4543-90c2-c83243a247a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:46.018262 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764cea2-9522-4543-90c2-c83243a247a5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:46.018901 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764cea2-9522-4543-90c2-c83243a247a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:46.019181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764cea2-9522-4543-90c2-c83243a247a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:46.020474 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f764cea2-9522-4543-90c2-c83243a247a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:46.021991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f764cea2-9522-4543-90c2-c83243a247a5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:46.023645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 18/43] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:34:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:34:46.024496 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764cea2-9522-4543-90c2-c83243a247a5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:46.083902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f2c21-2c50-40ab-94f9-9814b8211e89 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:46.138966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a759aa-f8b4-4c6b-828a-dc717d472130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:46.139709 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a759aa-f8b4-4c6b-828a-dc717d472130 None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:46.140195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a759aa-f8b4-4c6b-828a-dc717d472130 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:46.140609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a759aa-f8b4-4c6b-828a-dc717d472130 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:46.141352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a759aa-f8b4-4c6b-828a-dc717d472130 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:46.143487 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a759aa-f8b4-4c6b-828a-dc717d472130 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:46.163187 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 26/44] 10.222.0.36 () {60 vars in 1131 bytes} [Sat Oct 15 13:34:45 2022] POST /identity/v3/roles => generated 223 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:34:46.164521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a759aa-f8b4-4c6b-828a-dc717d472130 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:46.178293 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764cea2-9522-4543-90c2-c83243a247a5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:46.242889 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7997e1a-a5e9-4b34-a853-4d57af54df84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:46.243730 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7997e1a-a5e9-4b34-a853-4d57af54df84 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:46.244130 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7997e1a-a5e9-4b34-a853-4d57af54df84 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:46.244404 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7997e1a-a5e9-4b34-a853-4d57af54df84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:46.245219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7997e1a-a5e9-4b34-a853-4d57af54df84 None admin] RBAC: Adding query filter params (role.id=57740dc2ebe1429294f20dd51592b82a, scope.domain.id=default, user.id=d6c4e60be4154870948fd9be3f7e4091) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:34:46.245710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7997e1a-a5e9-4b34-a853-4d57af54df84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:46.248295 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7997e1a-a5e9-4b34-a853-4d57af54df84 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:46.267927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 19/45] 10.222.0.36 () {56 vars in 1322 bytes} [Sat Oct 15 13:34:46 2022] GET /identity/v3/role_assignments?user.id=d6c4e60be4154870948fd9be3f7e4091&scope.domain.id=default&role.id=57740dc2ebe1429294f20dd51592b82a => generated 530 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:46.335323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a759aa-f8b4-4c6b-828a-dc717d472130 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:46.353336 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:46.385897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3080e-cfc6-421e-8170-a9ecf1eaef63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:46.386757 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3080e-cfc6-421e-8170-a9ecf1eaef63 None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:46.387168 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3080e-cfc6-421e-8170-a9ecf1eaef63 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:46.387571 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3080e-cfc6-421e-8170-a9ecf1eaef63 None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:46.388468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3080e-cfc6-421e-8170-a9ecf1eaef63 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:46.390215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3080e-cfc6-421e-8170-a9ecf1eaef63 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:46.431418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 27/46] 10.222.0.36 () {60 vars in 1141 bytes} [Sat Oct 15 13:34:46 2022] POST /identity/v3/projects => generated 312 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:34:48.668734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61ef42-5ca2-49cc-ab37-1d0883eced85 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:48.669298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61ef42-5ca2-49cc-ab37-1d0883eced85 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:48.669298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61ef42-5ca2-49cc-ab37-1d0883eced85 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:48.670652 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 20/47] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:48.678261 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a311a7-c6c1-4f95-ab5f-b1bb19b13cef None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:48.678759 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a311a7-c6c1-4f95-ab5f-b1bb19b13cef None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:48.678843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a311a7-c6c1-4f95-ab5f-b1bb19b13cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:48.698907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-c9a311a7-c6c1-4f95-ab5f-b1bb19b13cef None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:48.700169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c9a311a7-c6c1-4f95-ab5f-b1bb19b13cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:48.883892 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 28/48] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:34:49.153363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61ef42-5ca2-49cc-ab37-1d0883eced85 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:49.179616 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:49.265711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61ef42-5ca2-49cc-ab37-1d0883eced85 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:49.304645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7162e-ab3b-4e70-8f61-b17b847c81e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:49.305330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7162e-ab3b-4e70-8f61-b17b847c81e4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:49.305630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7162e-ab3b-4e70-8f61-b17b847c81e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:49.305906 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7162e-ab3b-4e70-8f61-b17b847c81e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:49.307250 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7162e-ab3b-4e70-8f61-b17b847c81e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:49.309178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7162e-ab3b-4e70-8f61-b17b847c81e4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:49.310979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 21/49] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:34:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:49.317828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a311a7-c6c1-4f95-ab5f-b1bb19b13cef None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:49.435632 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:49.444806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a311a7-c6c1-4f95-ab5f-b1bb19b13cef None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:49.484962 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b589adc-9cd4-4f0d-abf5-f58b2502368c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:49.485975 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b589adc-9cd4-4f0d-abf5-f58b2502368c None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:49.486298 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b589adc-9cd4-4f0d-abf5-f58b2502368c None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:49.486612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b589adc-9cd4-4f0d-abf5-f58b2502368c None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:49.487498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b589adc-9cd4-4f0d-abf5-f58b2502368c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:49.489093 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b589adc-9cd4-4f0d-abf5-f58b2502368c None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:49.522616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 29/50] 10.222.0.36 () {60 vars in 1140 bytes} [Sat Oct 15 13:34:49 2022] POST /identity/v3/projects => generated 298 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:34:51.734751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd15136-9ab4-4566-9f83-ce37e6fd83c1 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:51.735348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd15136-9ab4-4566-9f83-ce37e6fd83c1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:51.735348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd15136-9ab4-4566-9f83-ce37e6fd83c1 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:51.737257 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 22/51] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:51.743373 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b92063-751d-49f3-9ebd-c967164d3946 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:51.743914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b92063-751d-49f3-9ebd-c967164d3946 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:51.744063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b92063-751d-49f3-9ebd-c967164d3946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:51.763416 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-d7b92063-751d-49f3-9ebd-c967164d3946 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:51.764799 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-d7b92063-751d-49f3-9ebd-c967164d3946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:51.841711 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 30/52] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:34:52.110093 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd15136-9ab4-4566-9f83-ce37e6fd83c1 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:52.209679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:52.221877 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd15136-9ab4-4566-9f83-ce37e6fd83c1 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:52.261341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69a8ef-9a99-4255-8a0d-16d56ddd5668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:52.262181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69a8ef-9a99-4255-8a0d-16d56ddd5668 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:52.262577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69a8ef-9a99-4255-8a0d-16d56ddd5668 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:52.262967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69a8ef-9a99-4255-8a0d-16d56ddd5668 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:52.264778 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69a8ef-9a99-4255-8a0d-16d56ddd5668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:52.266841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69a8ef-9a99-4255-8a0d-16d56ddd5668 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:52.269073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 23/53] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:34:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:52.275102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b92063-751d-49f3-9ebd-c967164d3946 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:52.370925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:52.401709 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b92063-751d-49f3-9ebd-c967164d3946 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:52.450585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61590d5-db74-483c-9cf4-dc5bdf4bb30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:52.451239 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61590d5-db74-483c-9cf4-dc5bdf4bb30a None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:52.451512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61590d5-db74-483c-9cf4-dc5bdf4bb30a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:52.451783 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61590d5-db74-483c-9cf4-dc5bdf4bb30a None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:52.452466 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61590d5-db74-483c-9cf4-dc5bdf4bb30a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:52.454010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61590d5-db74-483c-9cf4-dc5bdf4bb30a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:52.484694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 31/54] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:34:52 2022] POST /identity/v3/users => generated 275 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:34:56.295087 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8a5ba-b840-4d33-bf53-eef9fc14a879 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:56.296165 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8a5ba-b840-4d33-bf53-eef9fc14a879 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:56.296642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8a5ba-b840-4d33-bf53-eef9fc14a879 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:56.298691 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 24/55] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:56.304833 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11389e1-7408-446b-aa00-c9c037f0ce5f None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:56.305819 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11389e1-7408-446b-aa00-c9c037f0ce5f None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:56.306289 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11389e1-7408-446b-aa00-c9c037f0ce5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:56.333045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-f11389e1-7408-446b-aa00-c9c037f0ce5f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:56.334675 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f11389e1-7408-446b-aa00-c9c037f0ce5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:56.449877 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 32/56] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:34:56.524907 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33607baa-c982-4afb-84a5-723c8f2534a4 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:56.525559 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33607baa-c982-4afb-84a5-723c8f2534a4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:56.526030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33607baa-c982-4afb-84a5-723c8f2534a4 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:56.528099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 25/57] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:56.533630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ce390-693a-408e-a1a9-f33cada233a6 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:56.534460 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ce390-693a-408e-a1a9-f33cada233a6 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:56.535085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ce390-693a-408e-a1a9-f33cada233a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:56.565346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-4c7ce390-693a-408e-a1a9-f33cada233a6 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:56.567070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4c7ce390-693a-408e-a1a9-f33cada233a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:56.655815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:56.673435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 33/58] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:34:56.701285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1da01-85f4-43ee-8a62-660a95a24430 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:56.701606 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1da01-85f4-43ee-8a62-660a95a24430 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:56.701837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1da01-85f4-43ee-8a62-660a95a24430 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:56.703465 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 26/59] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:56.709650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48393f83-3cd0-48db-b5a5-bc85f13d5476 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:56.709941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48393f83-3cd0-48db-b5a5-bc85f13d5476 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:56.710223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48393f83-3cd0-48db-b5a5-bc85f13d5476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:56.738540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-48393f83-3cd0-48db-b5a5-bc85f13d5476 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:56.739765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-48393f83-3cd0-48db-b5a5-bc85f13d5476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:56.784992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1da01-85f4-43ee-8a62-660a95a24430 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:56.855905 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 34/60] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:34:56.963057 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48393f83-3cd0-48db-b5a5-bc85f13d5476 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:57.080702 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1da01-85f4-43ee-8a62-660a95a24430 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:57.107473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48393f83-3cd0-48db-b5a5-bc85f13d5476 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:57.134950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:57.154307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af138634-4562-45d4-af1d-b983d2d7aaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:57.155462 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af138634-4562-45d4-af1d-b983d2d7aaae None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:57.156066 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af138634-4562-45d4-af1d-b983d2d7aaae None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:57.156535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af138634-4562-45d4-af1d-b983d2d7aaae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:57.162904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39d466-caf6-44b9-9451-7d5f9935e0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:57.164257 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39d466-caf6-44b9-9451-7d5f9935e0d0 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:57.164930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39d466-caf6-44b9-9451-7d5f9935e0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:57.165419 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39d466-caf6-44b9-9451-7d5f9935e0d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:57.172002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af138634-4562-45d4-af1d-b983d2d7aaae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-af138634-4562-45d4-af1d-b983d2d7aaae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:34:57.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:34:57.180609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:34:57.181341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:34:57.181341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:34:57.181341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:34:57.181341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:34:57.181341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:34:57.181341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:34:57.181341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:34:57.181341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:34:57.181341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:34:57.181341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:34:57.181341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:34:57.181341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:34:57.181341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 27/61] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:34:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:34:57.183646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca77016-74d4-4cba-9a04-76766c9864bc None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:57.183963 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca77016-74d4-4cba-9a04-76766c9864bc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:57.184388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca77016-74d4-4cba-9a04-76766c9864bc None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:57.185737 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 28/62] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:57.186671 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca77016-74d4-4cba-9a04-76766c9864bc None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:57.190815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a39d466-caf6-44b9-9451-7d5f9935e0d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-7a39d466-caf6-44b9-9451-7d5f9935e0d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:34:57.196344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:34:57.197215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:34:57.197941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:34:57.197941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:34:57.197941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:34:57.197941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:34:57.197941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:34:57.197941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:34:57.197941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:34:57.197941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:34:57.197941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:34:57.197941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:34:57.197941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:34:57.197941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:34:57.197941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 35/63] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:34:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:34:57.198559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39d466-caf6-44b9-9451-7d5f9935e0d0 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:57.218905 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:57.313448 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca77016-74d4-4cba-9a04-76766c9864bc None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:57.315346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39d466-caf6-44b9-9451-7d5f9935e0d0 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:57.353373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a21c25-412d-41ef-9c27-a6be34945913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:57.354022 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a21c25-412d-41ef-9c27-a6be34945913 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:57.354289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a21c25-412d-41ef-9c27-a6be34945913 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:57.354417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a5962-71aa-4530-811d-8deaeadc8433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:57.354558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a21c25-412d-41ef-9c27-a6be34945913 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:57.354916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a5962-71aa-4530-811d-8deaeadc8433 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:57.355195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a5962-71aa-4530-811d-8deaeadc8433 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:57.355467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a5962-71aa-4530-811d-8deaeadc8433 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:57.356170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a5962-71aa-4530-811d-8deaeadc8433 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:34:57.356481 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a5962-71aa-4530-811d-8deaeadc8433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:57.357947 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a5962-71aa-4530-811d-8deaeadc8433 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:57.366544 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a21c25-412d-41ef-9c27-a6be34945913 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:34:57.367251 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 36/64] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:34:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:57.368550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e80182-824c-4bb2-93d4-4e61f20af660 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:57.368829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e80182-824c-4bb2-93d4-4e61f20af660 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:57.369098 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e80182-824c-4bb2-93d4-4e61f20af660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-80a21c25-412d-41ef-9c27-a6be34945913 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:34:57.371472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:34:57.373319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:34:57.374777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:34:57.374777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:34:57.374777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:34:57.374777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:34:57.374777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:34:57.374777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:34:57.374777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:34:57.374777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:34:57.374777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:34:57.374777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:34:57.374777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:34:57.374777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:34:57.374777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 29/65] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:34:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:34:57.374777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a21c25-412d-41ef-9c27-a6be34945913 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:57.394111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-07e80182-824c-4bb2-93d4-4e61f20af660 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:57.395371 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-07e80182-824c-4bb2-93d4-4e61f20af660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:57.439586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:57.458813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:57.470340 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 37/66] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:34:57.472927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820e2ff-b295-4b17-9160-f6bf240d9402 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:57.473414 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820e2ff-b295-4b17-9160-f6bf240d9402 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:57.473872 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820e2ff-b295-4b17-9160-f6bf240d9402 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:57.476284 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 38/67] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:34:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:34:57.477565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820e2ff-b295-4b17-9160-f6bf240d9402 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:57.490159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a21c25-412d-41ef-9c27-a6be34945913 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:57.538954 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabbcc6-572d-4064-83a9-4cc25ae49f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:57.539929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabbcc6-572d-4064-83a9-4cc25ae49f27 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:57.540428 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabbcc6-572d-4064-83a9-4cc25ae49f27 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:57.540853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabbcc6-572d-4064-83a9-4cc25ae49f27 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:57.541702 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabbcc6-572d-4064-83a9-4cc25ae49f27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:34:57.542163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabbcc6-572d-4064-83a9-4cc25ae49f27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:57.543901 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabbcc6-572d-4064-83a9-4cc25ae49f27 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:57.555937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 30/68] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:34:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:57.560097 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabbcc6-572d-4064-83a9-4cc25ae49f27 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:57.664080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820e2ff-b295-4b17-9160-f6bf240d9402 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:57.719625 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabbcc6-572d-4064-83a9-4cc25ae49f27 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:57.736253 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79608254-0d17-446a-9a82-453b8845508e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:57.737162 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79608254-0d17-446a-9a82-453b8845508e None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:57.737558 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79608254-0d17-446a-9a82-453b8845508e None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:57.737946 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79608254-0d17-446a-9a82-453b8845508e None admin] PATH_INFO: `/v3/users/d6c4e60be4154870948fd9be3f7e4091` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:57.739513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79608254-0d17-446a-9a82-453b8845508e None admin] RBAC: Authorizing `identity:get_user(user_id=d6c4e60be4154870948fd9be3f7e4091)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:57.741365 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79608254-0d17-446a-9a82-453b8845508e None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:57.743342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 39/69] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:34:57 2022] GET /identity/v3/users/d6c4e60be4154870948fd9be3f7e4091 => generated 247 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:57.747952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affd876-cae4-4049-8ba0-745ab85df5dc None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:57.747952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affd876-cae4-4049-8ba0-745ab85df5dc None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:57.747952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affd876-cae4-4049-8ba0-745ab85df5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:57.770602 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-1affd876-cae4-4049-8ba0-745ab85df5dc None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:34:57.772063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1affd876-cae4-4049-8ba0-745ab85df5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:34:57.784769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec812cd-f6aa-415e-be66-973d5ca5e725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:57.786356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec812cd-f6aa-415e-be66-973d5ca5e725 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:57.786811 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec812cd-f6aa-415e-be66-973d5ca5e725 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:57.787204 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec812cd-f6aa-415e-be66-973d5ca5e725 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:57.788089 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec812cd-f6aa-415e-be66-973d5ca5e725 None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:34:57.788515 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec812cd-f6aa-415e-be66-973d5ca5e725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:57.790536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec812cd-f6aa-415e-be66-973d5ca5e725 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:57.804158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 31/70] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:34:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:57.805207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec812cd-f6aa-415e-be66-973d5ca5e725 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:57.809027 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:57.823829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:57.859950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 40/71] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:34:57.861719 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affd876-cae4-4049-8ba0-745ab85df5dc None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:57.971779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec812cd-f6aa-415e-be66-973d5ca5e725 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:58.030986 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affd876-cae4-4049-8ba0-745ab85df5dc None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:58.033798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b2b40-a42d-43c7-b12f-958f803827b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:58.034813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b2b40-a42d-43c7-b12f-958f803827b5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:58.035290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b2b40-a42d-43c7-b12f-958f803827b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:58.035755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b2b40-a42d-43c7-b12f-958f803827b5 None admin] PATH_INFO: `/v3/users/852f3d41db194fdcbd8c6e771be4b167` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:58.061804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b2b40-a42d-43c7-b12f-958f803827b5 None admin] RBAC: Authorizing `identity:get_user(user_id=852f3d41db194fdcbd8c6e771be4b167)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:58.064681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b2b40-a42d-43c7-b12f-958f803827b5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:58.068358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 32/72] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:34:57 2022] GET /identity/v3/users/852f3d41db194fdcbd8c6e771be4b167 => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:58.070702 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9193b75-567b-434e-a6fe-94c2eda4422a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:58.071357 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9193b75-567b-434e-a6fe-94c2eda4422a None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:58.071631 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9193b75-567b-434e-a6fe-94c2eda4422a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:58.071996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9193b75-567b-434e-a6fe-94c2eda4422a None admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:58.073177 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9193b75-567b-434e-a6fe-94c2eda4422a None admin] RBAC: Authorizing `identity:get_project(project_id=b3780a3b8fe742949173e4396724f2e0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:58.074270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b2b40-a42d-43c7-b12f-958f803827b5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:58.074645 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9193b75-567b-434e-a6fe-94c2eda4422a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:58.076666 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 41/73] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:34:57 2022] GET /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:58.077443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9193b75-567b-434e-a6fe-94c2eda4422a None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:58.128244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:58.181999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:58.225159 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9193b75-567b-434e-a6fe-94c2eda4422a None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:58.250934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b2b40-a42d-43c7-b12f-958f803827b5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:58.265420 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5b0a6-4eaa-4dc9-b947-9b0da19ca663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:58.266063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5b0a6-4eaa-4dc9-b947-9b0da19ca663 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:58.266337 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5b0a6-4eaa-4dc9-b947-9b0da19ca663 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:58.266618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5b0a6-4eaa-4dc9-b947-9b0da19ca663 None admin] PATH_INFO: `/v3/users/852f3d41db194fdcbd8c6e771be4b167` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:58.268290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5b0a6-4eaa-4dc9-b947-9b0da19ca663 None admin] RBAC: Authorizing `identity:get_user(user_id=852f3d41db194fdcbd8c6e771be4b167)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:58.270180 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5b0a6-4eaa-4dc9-b947-9b0da19ca663 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:58.279827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 42/74] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:34:58 2022] GET /identity/v3/users/852f3d41db194fdcbd8c6e771be4b167 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:58.279827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5b0a6-4eaa-4dc9-b947-9b0da19ca663 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:58.312639 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bc1c7-b7f8-438e-9a92-d7f6c1457175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:58.313429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bc1c7-b7f8-438e-9a92-d7f6c1457175 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:58.313785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bc1c7-b7f8-438e-9a92-d7f6c1457175 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:58.314661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bc1c7-b7f8-438e-9a92-d7f6c1457175 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:58.327769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661bc1c7-b7f8-438e-9a92-d7f6c1457175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-661bc1c7-b7f8-438e-9a92-d7f6c1457175 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:34:58.333618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:34:58.335512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:34:58.337259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:34:58.337259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:34:58.337259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:34:58.337259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:34:58.337259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:34:58.337259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:34:58.337259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:34:58.337259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:34:58.337259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:34:58.337259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:34:58.337259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:34:58.337259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:34:58.337259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 33/75] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:34:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:34:58.337259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bc1c7-b7f8-438e-9a92-d7f6c1457175 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:58.412621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5b0a6-4eaa-4dc9-b947-9b0da19ca663 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:58.424688 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:58.457295 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea2fcb-5f84-48b9-a3ed-9a237f5770d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:58.458109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea2fcb-5f84-48b9-a3ed-9a237f5770d6 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:58.458503 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea2fcb-5f84-48b9-a3ed-9a237f5770d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:58.458877 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea2fcb-5f84-48b9-a3ed-9a237f5770d6 None admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:58.460467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea2fcb-5f84-48b9-a3ed-9a237f5770d6 None admin] RBAC: Authorizing `identity:get_project(project_id=b3780a3b8fe742949173e4396724f2e0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:58.462250 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea2fcb-5f84-48b9-a3ed-9a237f5770d6 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:58.464342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 43/76] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:34:58 2022] GET /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:58.465409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea2fcb-5f84-48b9-a3ed-9a237f5770d6 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:58.468959 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bc1c7-b7f8-438e-9a92-d7f6c1457175 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:58.500352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:58.518846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f7bb1-3c62-41ea-bfe5-68fa8d062118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:58.519970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f7bb1-3c62-41ea-bfe5-68fa8d062118 None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:58.520366 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f7bb1-3c62-41ea-bfe5-68fa8d062118 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:58.520706 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f7bb1-3c62-41ea-bfe5-68fa8d062118 None admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0/users/d6c4e60be4154870948fd9be3f7e4091/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:58.522888 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f7bb1-3c62-41ea-bfe5-68fa8d062118 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3780a3b8fe742949173e4396724f2e0, user_id=d6c4e60be4154870948fd9be3f7e4091, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:58.524525 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f7bb1-3c62-41ea-bfe5-68fa8d062118 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:58.542420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 34/77] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:34:58 2022] PUT /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0/users/d6c4e60be4154870948fd9be3f7e4091/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:34:58.543911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f7bb1-3c62-41ea-bfe5-68fa8d062118 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:58.614089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea2fcb-5f84-48b9-a3ed-9a237f5770d6 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:58.660865 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49982a-090f-455e-9a49-607e0a918a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:58.661977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49982a-090f-455e-9a49-607e0a918a9a None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:58.662464 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49982a-090f-455e-9a49-607e0a918a9a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:58.662907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49982a-090f-455e-9a49-607e0a918a9a None admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:58.664594 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49982a-090f-455e-9a49-607e0a918a9a None admin] RBAC: Authorizing `identity:get_project(project_id=b3780a3b8fe742949173e4396724f2e0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:58.666954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49982a-090f-455e-9a49-607e0a918a9a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:58.669025 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 44/78] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:34:58 2022] GET /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:58.670723 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49982a-090f-455e-9a49-607e0a918a9a None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:58.709974 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f7bb1-3c62-41ea-bfe5-68fa8d062118 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:58.730047 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:58.764569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5325735-e894-4af5-9a75-c457e8e1896e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:58.765144 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5325735-e894-4af5-9a75-c457e8e1896e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:58.765421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5325735-e894-4af5-9a75-c457e8e1896e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:58.765699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5325735-e894-4af5-9a75-c457e8e1896e None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:58.766882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5325735-e894-4af5-9a75-c457e8e1896e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:58.768466 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5325735-e894-4af5-9a75-c457e8e1896e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:58.770088 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 35/79] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:34:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:58.770848 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5325735-e894-4af5-9a75-c457e8e1896e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:58.811448 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49982a-090f-455e-9a49-607e0a918a9a None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:58.834788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:58.883582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a029e87-2afa-44a6-9ab1-947346d8b6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:58.887197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a029e87-2afa-44a6-9ab1-947346d8b6e2 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:58.887197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a029e87-2afa-44a6-9ab1-947346d8b6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:58.887197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a029e87-2afa-44a6-9ab1-947346d8b6e2 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:58.887197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a029e87-2afa-44a6-9ab1-947346d8b6e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:34:58.887570 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a029e87-2afa-44a6-9ab1-947346d8b6e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:58.889945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a029e87-2afa-44a6-9ab1-947346d8b6e2 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:58.902861 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 45/80] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:34:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:58.903720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a029e87-2afa-44a6-9ab1-947346d8b6e2 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:58.912917 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5325735-e894-4af5-9a75-c457e8e1896e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:58.977095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0dac3-6d9f-4a25-a53b-b4b71adae57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:58.978648 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0dac3-6d9f-4a25-a53b-b4b71adae57d None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:58.979179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0dac3-6d9f-4a25-a53b-b4b71adae57d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:58.979650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0dac3-6d9f-4a25-a53b-b4b71adae57d None admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0/users/852f3d41db194fdcbd8c6e771be4b167/roles/8a864244c21042b88accaa76b9117665` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:58.983555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0dac3-6d9f-4a25-a53b-b4b71adae57d None admin] RBAC: Authorizing `identity:create_grant(project_id=b3780a3b8fe742949173e4396724f2e0, user_id=852f3d41db194fdcbd8c6e771be4b167, role_id=8a864244c21042b88accaa76b9117665)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:58.986820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0dac3-6d9f-4a25-a53b-b4b71adae57d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:59.009987 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 36/81] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:34:58 2022] PUT /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0/users/852f3d41db194fdcbd8c6e771be4b167/roles/8a864244c21042b88accaa76b9117665 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:34:59.020336 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0dac3-6d9f-4a25-a53b-b4b71adae57d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:59.063846 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a029e87-2afa-44a6-9ab1-947346d8b6e2 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:59.080160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:59.136823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7e7a0-674a-4919-88e1-12caadc85ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:59.137925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7e7a0-674a-4919-88e1-12caadc85ae8 None admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:59.138392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7e7a0-674a-4919-88e1-12caadc85ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:59.138856 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7e7a0-674a-4919-88e1-12caadc85ae8 None admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0/users/852f3d41db194fdcbd8c6e771be4b167/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:59.142751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7e7a0-674a-4919-88e1-12caadc85ae8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3780a3b8fe742949173e4396724f2e0, user_id=852f3d41db194fdcbd8c6e771be4b167, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:59.145888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7e7a0-674a-4919-88e1-12caadc85ae8 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:59.152898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0dac3-6d9f-4a25-a53b-b4b71adae57d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:59.165476 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:59.179539 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 46/82] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:34:58 2022] PUT /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0/users/852f3d41db194fdcbd8c6e771be4b167/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:34:59.186891 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7e7a0-674a-4919-88e1-12caadc85ae8 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:59.199004 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39028f6d-40c9-4b8e-be84-5d62b6b71c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:59.200166 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39028f6d-40c9-4b8e-be84-5d62b6b71c30 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:59.200704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39028f6d-40c9-4b8e-be84-5d62b6b71c30 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:59.201255 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39028f6d-40c9-4b8e-be84-5d62b6b71c30 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:59.202200 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39028f6d-40c9-4b8e-be84-5d62b6b71c30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:59.204012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39028f6d-40c9-4b8e-be84-5d62b6b71c30 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:59.274532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 37/83] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:34:59 2022] POST /identity/v3/users => generated 289 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:34:59.410555 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7e7a0-674a-4919-88e1-12caadc85ae8 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:59.475154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd994a-6968-4347-8052-4ca4d17c01b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:59.476269 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd994a-6968-4347-8052-4ca4d17c01b1 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:59.476740 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd994a-6968-4347-8052-4ca4d17c01b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:59.477204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd994a-6968-4347-8052-4ca4d17c01b1 None admin] PATH_INFO: `/v3/users/852f3d41db194fdcbd8c6e771be4b167` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:59.479303 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd994a-6968-4347-8052-4ca4d17c01b1 None admin] RBAC: Authorizing `identity:get_user(user_id=852f3d41db194fdcbd8c6e771be4b167)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:59.482070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd994a-6968-4347-8052-4ca4d17c01b1 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:59.485800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 47/84] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:34:59 2022] GET /identity/v3/users/852f3d41db194fdcbd8c6e771be4b167 => generated 275 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:59.492107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39028f6d-40c9-4b8e-be84-5d62b6b71c30 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:59.678001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39028f6d-40c9-4b8e-be84-5d62b6b71c30 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:59.748190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f36ad-4a90-439b-ac57-428020d4a681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:59.749354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f36ad-4a90-439b-ac57-428020d4a681 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:59.749854 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f36ad-4a90-439b-ac57-428020d4a681 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:59.750351 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f36ad-4a90-439b-ac57-428020d4a681 None admin] PATH_INFO: `/v3/projects/c3903475dd6549c19154391aacc9b814` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:59.753138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f36ad-4a90-439b-ac57-428020d4a681 None admin] RBAC: Authorizing `identity:get_project(project_id=c3903475dd6549c19154391aacc9b814)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:59.756152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f36ad-4a90-439b-ac57-428020d4a681 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:59.759260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 38/85] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:34:59 2022] GET /identity/v3/projects/c3903475dd6549c19154391aacc9b814 => generated 312 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:34:59.766072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd994a-6968-4347-8052-4ca4d17c01b1 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:34:59.811496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:34:59.924477 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd994a-6968-4347-8052-4ca4d17c01b1 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:34:59.967963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055f6a2-5037-4b4d-bb65-8dec47fd3eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:34:59.968599 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055f6a2-5037-4b4d-bb65-8dec47fd3eca None admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:34:59.968878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055f6a2-5037-4b4d-bb65-8dec47fd3eca None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:34:59.969153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055f6a2-5037-4b4d-bb65-8dec47fd3eca None admin] PATH_INFO: `/v3/projects/c3903475dd6549c19154391aacc9b814/users/852f3d41db194fdcbd8c6e771be4b167/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:34:59.971423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055f6a2-5037-4b4d-bb65-8dec47fd3eca None admin] RBAC: Authorizing `identity:create_grant(project_id=c3903475dd6549c19154391aacc9b814, user_id=852f3d41db194fdcbd8c6e771be4b167, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:34:59.973063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055f6a2-5037-4b4d-bb65-8dec47fd3eca None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:34:59.993524 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 48/86] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:34:59 2022] PUT /identity/v3/projects/c3903475dd6549c19154391aacc9b814/users/852f3d41db194fdcbd8c6e771be4b167/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:35:01.832227 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bef58e-9f2b-41a9-8242-079141d94a57 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:01.833296 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bef58e-9f2b-41a9-8242-079141d94a57 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:01.833786 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bef58e-9f2b-41a9-8242-079141d94a57 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:01.835359 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 39/87] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:01.841243 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096623f-4e16-4bd2-a161-3abd4d5e9282 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:01.842149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096623f-4e16-4bd2-a161-3abd4d5e9282 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:01.842641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096623f-4e16-4bd2-a161-3abd4d5e9282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:01.868959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-2096623f-4e16-4bd2-a161-3abd4d5e9282 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:01.870552 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2096623f-4e16-4bd2-a161-3abd4d5e9282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:01.954724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 49/88] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:02.445062 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f40054-0888-4f7d-905f-36f607b9edf9 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:02.445763 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f40054-0888-4f7d-905f-36f607b9edf9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:02.446232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f40054-0888-4f7d-905f-36f607b9edf9 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:02.447805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 40/89] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:02.453832 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7b293-2227-4b36-905c-a49fb51fe589 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:02.454148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7b293-2227-4b36-905c-a49fb51fe589 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:02.454412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7b293-2227-4b36-905c-a49fb51fe589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:02.485640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-ceb7b293-2227-4b36-905c-a49fb51fe589 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:02.486951 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ceb7b293-2227-4b36-905c-a49fb51fe589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:02.504124 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f40054-0888-4f7d-905f-36f607b9edf9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:02.506932 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:02.587802 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:02.645800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 50/90] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:02.647158 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2dd51-df37-4ef5-aed4-8e4855592525 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:02.647447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2dd51-df37-4ef5-aed4-8e4855592525 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:02.647724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2dd51-df37-4ef5-aed4-8e4855592525 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:02.649286 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 51/91] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:02.656691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ccf2f-0a45-4629-b828-8038ab2c05c9 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:02.656979 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ccf2f-0a45-4629-b828-8038ab2c05c9 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:02.657275 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ccf2f-0a45-4629-b828-8038ab2c05c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:02.688955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-ce7ccf2f-0a45-4629-b828-8038ab2c05c9 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:02.690585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ce7ccf2f-0a45-4629-b828-8038ab2c05c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:02.756430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f40054-0888-4f7d-905f-36f607b9edf9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:02.816836 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 52/92] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:02.837961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075395d-a771-47c5-96ed-69bc634523e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:02.838611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075395d-a771-47c5-96ed-69bc634523e4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:02.838891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075395d-a771-47c5-96ed-69bc634523e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:02.839166 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075395d-a771-47c5-96ed-69bc634523e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:02.861156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075395d-a771-47c5-96ed-69bc634523e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-6075395d-a771-47c5-96ed-69bc634523e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:02.866742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:02.867600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:02.868376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:02.868376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:02.868376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:02.868376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:02.868376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:02.868376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:02.868376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:02.868376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:02.868376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:02.868376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:02.868376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:35:02.868376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:02.871962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 41/93] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:35:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:02.876485 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ccf2f-0a45-4629-b828-8038ab2c05c9 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:03.015388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:03.036034 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ccf2f-0a45-4629-b828-8038ab2c05c9 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:03.050204 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075395d-a771-47c5-96ed-69bc634523e4 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:03.091735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab1866-8c31-4553-ae4a-052376f625e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:03.092698 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab1866-8c31-4553-ae4a-052376f625e3 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:03.093144 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab1866-8c31-4553-ae4a-052376f625e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:03.093611 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab1866-8c31-4553-ae4a-052376f625e3 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:03.094628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab1866-8c31-4553-ae4a-052376f625e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:03.095118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab1866-8c31-4553-ae4a-052376f625e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:03.096777 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab1866-8c31-4553-ae4a-052376f625e3 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:03.107925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 53/94] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:03.117012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab1866-8c31-4553-ae4a-052376f625e3 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:03.236318 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab1866-8c31-4553-ae4a-052376f625e3 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:03.273386 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:03.287146 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075395d-a771-47c5-96ed-69bc634523e4 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:03.325755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc0c94-cc65-4138-bd65-4909ef53a12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:03.326813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc0c94-cc65-4138-bd65-4909ef53a12c None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:03.327285 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc0c94-cc65-4138-bd65-4909ef53a12c None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:03.327774 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc0c94-cc65-4138-bd65-4909ef53a12c None admin] PATH_INFO: `/v3/users/d6c4e60be4154870948fd9be3f7e4091` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:03.329258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc0c94-cc65-4138-bd65-4909ef53a12c None admin] RBAC: Authorizing `identity:get_user(user_id=d6c4e60be4154870948fd9be3f7e4091)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:03.331525 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc0c94-cc65-4138-bd65-4909ef53a12c None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:03.333515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 54/95] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:03 2022] GET /identity/v3/users/d6c4e60be4154870948fd9be3f7e4091 => generated 247 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:03.338984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f67635-be5d-4f74-91c8-86d8c4cd57a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:03.340155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f67635-be5d-4f74-91c8-86d8c4cd57a5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:03.340704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f67635-be5d-4f74-91c8-86d8c4cd57a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:03.340938 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc0c94-cc65-4138-bd65-4909ef53a12c None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:03.341454 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f67635-be5d-4f74-91c8-86d8c4cd57a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:03.360897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f67635-be5d-4f74-91c8-86d8c4cd57a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-66f67635-be5d-4f74-91c8-86d8c4cd57a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:03.369431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:03.376516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:03.378090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:03.378090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:03.378090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:03.378090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:03.378090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:03.378090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:03.378090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:03.378090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:03.378090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:03.378090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:03.378090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:03.378090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:03.378090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 42/96] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:35:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:35:03.378873 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:03.380978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f67635-be5d-4f74-91c8-86d8c4cd57a5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:03.556502 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc0c94-cc65-4138-bd65-4909ef53a12c None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:03.588053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f67635-be5d-4f74-91c8-86d8c4cd57a5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:03.636767 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929e20f-d6f2-4070-9d57-c4e313157a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:03.637642 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929e20f-d6f2-4070-9d57-c4e313157a86 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:03.637923 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929e20f-d6f2-4070-9d57-c4e313157a86 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:03.638252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929e20f-d6f2-4070-9d57-c4e313157a86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:03.665980 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f24295-b73a-4ad7-a31f-16d064770f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:03.666629 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f24295-b73a-4ad7-a31f-16d064770f29 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:03.666911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f24295-b73a-4ad7-a31f-16d064770f29 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:03.667175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f24295-b73a-4ad7-a31f-16d064770f29 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:03.667911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f24295-b73a-4ad7-a31f-16d064770f29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:03.668224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f24295-b73a-4ad7-a31f-16d064770f29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:03.670131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f24295-b73a-4ad7-a31f-16d064770f29 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:03.687057 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 43/97] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:35:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:03.687987 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f24295-b73a-4ad7-a31f-16d064770f29 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:03.726164 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:03.771661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929e20f-d6f2-4070-9d57-c4e313157a86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:03.774549 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929e20f-d6f2-4070-9d57-c4e313157a86 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:03.801142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:03.843607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f24295-b73a-4ad7-a31f-16d064770f29 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:03.899117 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0346cb-592e-4fc0-bc1c-50b432a7985b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:03.900288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0346cb-592e-4fc0-bc1c-50b432a7985b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:03.900580 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0346cb-592e-4fc0-bc1c-50b432a7985b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:03.900955 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0346cb-592e-4fc0-bc1c-50b432a7985b None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:03.917225 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0346cb-592e-4fc0-bc1c-50b432a7985b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:03.924244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 55/98] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:35:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:35:03.924244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929e20f-d6f2-4070-9d57-c4e313157a86 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-3a0346cb-592e-4fc0-bc1c-50b432a7985b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:03.925189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:03.926107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:03.927084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:03.927084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:03.927084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:03.927084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:03.927084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:03.927084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:03.927084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:03.927084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:03.927084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:03.927084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:03.927084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:03.927084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:03.927084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 44/99] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:03.927084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8135e-77e4-481e-857a-ff47cd1d5467 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:03.927829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8135e-77e4-481e-857a-ff47cd1d5467 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:03.927829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8135e-77e4-481e-857a-ff47cd1d5467 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:03.931490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 45/100] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:03.931490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd157b22-fbdd-4376-9805-b4514d1d7a4a None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:03.931490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd157b22-fbdd-4376-9805-b4514d1d7a4a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:03.931490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd157b22-fbdd-4376-9805-b4514d1d7a4a None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:03.932290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 46/101] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:03.935995 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd157b22-fbdd-4376-9805-b4514d1d7a4a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:04.054898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:04.069001 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929e20f-d6f2-4070-9d57-c4e313157a86 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:04.086434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd157b22-fbdd-4376-9805-b4514d1d7a4a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:04.087001 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:04.115350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2d682-41b7-4c2a-87a5-19f7d0a3f4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:04.116056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2d682-41b7-4c2a-87a5-19f7d0a3f4b8 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:04.116341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2d682-41b7-4c2a-87a5-19f7d0a3f4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:04.116617 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2d682-41b7-4c2a-87a5-19f7d0a3f4b8 None admin] PATH_INFO: `/v3/users/852f3d41db194fdcbd8c6e771be4b167` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:04.117896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2d682-41b7-4c2a-87a5-19f7d0a3f4b8 None admin] RBAC: Authorizing `identity:get_user(user_id=852f3d41db194fdcbd8c6e771be4b167)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:04.119404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2d682-41b7-4c2a-87a5-19f7d0a3f4b8 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:04.121178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 56/102] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:03 2022] GET /identity/v3/users/852f3d41db194fdcbd8c6e771be4b167 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:04.122034 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2d682-41b7-4c2a-87a5-19f7d0a3f4b8 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:04.139683 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f66969-940c-4776-a1cf-bdc017f1fbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:04.140699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f66969-940c-4776-a1cf-bdc017f1fbe7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:04.141091 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f66969-940c-4776-a1cf-bdc017f1fbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:04.141533 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f66969-940c-4776-a1cf-bdc017f1fbe7 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:04.142655 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f66969-940c-4776-a1cf-bdc017f1fbe7 None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:04.144014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f66969-940c-4776-a1cf-bdc017f1fbe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:04.146444 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f66969-940c-4776-a1cf-bdc017f1fbe7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:04.158114 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 47/103] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:04.160603 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38447d18-a5a6-4667-87fc-295db6d3aaeb None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:04.161608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38447d18-a5a6-4667-87fc-295db6d3aaeb None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:04.161608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38447d18-a5a6-4667-87fc-295db6d3aaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:04.184070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-38447d18-a5a6-4667-87fc-295db6d3aaeb None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:04.185350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-38447d18-a5a6-4667-87fc-295db6d3aaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:04.262267 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2d682-41b7-4c2a-87a5-19f7d0a3f4b8 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:04.272090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 48/104] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:04.273464 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e205ca-f8ef-4a33-9001-8ed4d26dd9b9 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:04.273738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e205ca-f8ef-4a33-9001-8ed4d26dd9b9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:04.274014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e205ca-f8ef-4a33-9001-8ed4d26dd9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:04.295169 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-54e205ca-f8ef-4a33-9001-8ed4d26dd9b9 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:04.297232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-54e205ca-f8ef-4a33-9001-8ed4d26dd9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:04.306920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed60c2b-b02c-4fc6-a89f-8858081be8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:04.307596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed60c2b-b02c-4fc6-a89f-8858081be8a4 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:04.307903 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed60c2b-b02c-4fc6-a89f-8858081be8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:04.308289 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed60c2b-b02c-4fc6-a89f-8858081be8a4 None admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:04.309660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed60c2b-b02c-4fc6-a89f-8858081be8a4 None admin] RBAC: Authorizing `identity:get_project(project_id=b3780a3b8fe742949173e4396724f2e0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:04.311160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed60c2b-b02c-4fc6-a89f-8858081be8a4 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:04.313406 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 57/105] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:04 2022] GET /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:04.314788 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c75b9-e8c1-4a64-9b5f-11ac2f4ec968 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:04.315453 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c75b9-e8c1-4a64-9b5f-11ac2f4ec968 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:04.315746 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c75b9-e8c1-4a64-9b5f-11ac2f4ec968 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:04.317446 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 58/106] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:04.318022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c75b9-e8c1-4a64-9b5f-11ac2f4ec968 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:04.392303 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:04.407038 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:04.415605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 49/107] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:04.416492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e205ca-f8ef-4a33-9001-8ed4d26dd9b9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:04.486575 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c75b9-e8c1-4a64-9b5f-11ac2f4ec968 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:04.534924 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25be10e-c969-4d64-94ce-88a281984da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:04.536101 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25be10e-c969-4d64-94ce-88a281984da2 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:04.537085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25be10e-c969-4d64-94ce-88a281984da2 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:04.537562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25be10e-c969-4d64-94ce-88a281984da2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:04.568567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e205ca-f8ef-4a33-9001-8ed4d26dd9b9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:04.624325 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25be10e-c969-4d64-94ce-88a281984da2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:04.626399 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25be10e-c969-4d64-94ce-88a281984da2 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:04.628958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7959f-9a48-4a97-8a1a-cf397c502d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:04.630763 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7959f-9a48-4a97-8a1a-cf397c502d38 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:04.630763 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7959f-9a48-4a97-8a1a-cf397c502d38 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:04.631290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7959f-9a48-4a97-8a1a-cf397c502d38 None admin] PATH_INFO: `/v3/users/852f3d41db194fdcbd8c6e771be4b167` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:04.633396 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7959f-9a48-4a97-8a1a-cf397c502d38 None admin] RBAC: Authorizing `identity:get_user(user_id=852f3d41db194fdcbd8c6e771be4b167)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:04.636105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7959f-9a48-4a97-8a1a-cf397c502d38 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:04.639374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 50/108] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:04 2022] GET /identity/v3/users/852f3d41db194fdcbd8c6e771be4b167 => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:04.641151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7959f-9a48-4a97-8a1a-cf397c502d38 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:04.687243 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:04.736344 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:04.749640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 59/109] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:35:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:35:04.751029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d865d-39d2-4362-a9f7-70b217c5f2af None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:04.751320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d865d-39d2-4362-a9f7-70b217c5f2af None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:04.751597 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d865d-39d2-4362-a9f7-70b217c5f2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:04.771352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-564d865d-39d2-4362-a9f7-70b217c5f2af None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:04.772624 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-564d865d-39d2-4362-a9f7-70b217c5f2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:04.785770 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7959f-9a48-4a97-8a1a-cf397c502d38 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:04.830365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e230f2-bf26-429e-9cd7-95076f5247f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:04.831433 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e230f2-bf26-429e-9cd7-95076f5247f1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:04.831935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e230f2-bf26-429e-9cd7-95076f5247f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:04.832425 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e230f2-bf26-429e-9cd7-95076f5247f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:04.833787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e230f2-bf26-429e-9cd7-95076f5247f1 None admin] RBAC: Adding query filter params (role.id=57740dc2ebe1429294f20dd51592b82a, scope.project.id=b3780a3b8fe742949173e4396724f2e0, user.id=d6c4e60be4154870948fd9be3f7e4091) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:04.834338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e230f2-bf26-429e-9cd7-95076f5247f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:04.837017 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e230f2-bf26-429e-9cd7-95076f5247f1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:04.844811 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 60/110] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:04.845726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d865d-39d2-4362-a9f7-70b217c5f2af None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:04.848727 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 51/111] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:35:04 2022] GET /identity/v3/role_assignments?user.id=d6c4e60be4154870948fd9be3f7e4091&scope.project.id=b3780a3b8fe742949173e4396724f2e0&role.id=57740dc2ebe1429294f20dd51592b82a => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:04.860927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e230f2-bf26-429e-9cd7-95076f5247f1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:04.978164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:05.004740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e230f2-bf26-429e-9cd7-95076f5247f1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:05.011789 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d865d-39d2-4362-a9f7-70b217c5f2af None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:05.043335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:05.056716 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783a3b1-ed87-4a74-93db-7a1ce60e5b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:05.057788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783a3b1-ed87-4a74-93db-7a1ce60e5b43 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:05.058143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783a3b1-ed87-4a74-93db-7a1ce60e5b43 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:05.058499 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783a3b1-ed87-4a74-93db-7a1ce60e5b43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:05.067621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20658a24-1790-4284-9149-a17649db7d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:05.069035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20658a24-1790-4284-9149-a17649db7d4f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:05.069510 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20658a24-1790-4284-9149-a17649db7d4f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:05.070449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20658a24-1790-4284-9149-a17649db7d4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:05.073705 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783a3b1-ed87-4a74-93db-7a1ce60e5b43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-c783a3b1-ed87-4a74-93db-7a1ce60e5b43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:05.080830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:05.081747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:05.082482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:05.082482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:05.082482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:05.082482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:05.082482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:05.082482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:05.082482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:05.082482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:05.082482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:05.082482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:05.082482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:05.082482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:05.085135 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 52/112] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:05.086247 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783a3b1-ed87-4a74-93db-7a1ce60e5b43 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:05.173630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20658a24-1790-4284-9149-a17649db7d4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:05.175517 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20658a24-1790-4284-9149-a17649db7d4f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:05.200806 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783a3b1-ed87-4a74-93db-7a1ce60e5b43 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:05.270493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7eee2-744f-45c5-9dde-f136022b5db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:05.271528 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7eee2-744f-45c5-9dde-f136022b5db1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:05.271853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7eee2-744f-45c5-9dde-f136022b5db1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:05.272158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7eee2-744f-45c5-9dde-f136022b5db1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:05.290496 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7eee2-744f-45c5-9dde-f136022b5db1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-ecb7eee2-744f-45c5-9dde-f136022b5db1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:05.298534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:05.300313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:05.301885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:05.301885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:05.301885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:05.301885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:05.301885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:05.301885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:05.301885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:05.301885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:05.301885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:05.301885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:05.301885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 13:35:05.301885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:05.301885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 53/113] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:05.309650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7eee2-744f-45c5-9dde-f136022b5db1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:05.320556 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:05.342157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 61/114] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:35:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:35:05.343218 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20658a24-1790-4284-9149-a17649db7d4f None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:05.398637 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:05.466355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20658a24-1790-4284-9149-a17649db7d4f None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:05.477091 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7eee2-744f-45c5-9dde-f136022b5db1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:05.520704 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd00fbe-d805-4fe8-a089-14230994ae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:05.521604 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd00fbe-d805-4fe8-a089-14230994ae0a None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:05.522072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd00fbe-d805-4fe8-a089-14230994ae0a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:05.522542 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd00fbe-d805-4fe8-a089-14230994ae0a None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:05.523673 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd00fbe-d805-4fe8-a089-14230994ae0a None admin] RBAC: Adding query filter params (name=member) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:05.524327 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd00fbe-d805-4fe8-a089-14230994ae0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:05.525248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d943a-69b5-4efe-a3b0-ba013168bd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:05.526222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d943a-69b5-4efe-a3b0-ba013168bd3e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:05.526645 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd00fbe-d805-4fe8-a089-14230994ae0a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:05.526744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d943a-69b5-4efe-a3b0-ba013168bd3e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:05.527074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d943a-69b5-4efe-a3b0-ba013168bd3e None admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:05.528869 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d943a-69b5-4efe-a3b0-ba013168bd3e None admin] RBAC: Authorizing `identity:get_project(project_id=b3780a3b8fe742949173e4396724f2e0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:05.531347 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d943a-69b5-4efe-a3b0-ba013168bd3e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:05.534568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 54/115] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:05 2022] GET /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:05.535423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d943a-69b5-4efe-a3b0-ba013168bd3e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:05.540676 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 62/116] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:05.542091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd00fbe-d805-4fe8-a089-14230994ae0a None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:05.616128 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:05.659903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d943a-69b5-4efe-a3b0-ba013168bd3e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:05.677714 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:05.678249 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd00fbe-d805-4fe8-a089-14230994ae0a None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:05.698400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c94304-ae79-4e50-a78b-ebf5e505956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:05.699145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c94304-ae79-4e50-a78b-ebf5e505956e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:05.699500 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c94304-ae79-4e50-a78b-ebf5e505956e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:05.699990 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c94304-ae79-4e50-a78b-ebf5e505956e None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:05.700779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c94304-ae79-4e50-a78b-ebf5e505956e None admin] RBAC: Adding query filter params (name=reader) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:05.701128 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c94304-ae79-4e50-a78b-ebf5e505956e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:05.702763 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c94304-ae79-4e50-a78b-ebf5e505956e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:05.712631 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 55/117] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:05.713738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c94304-ae79-4e50-a78b-ebf5e505956e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:05.715352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b23eca-0d43-42c7-a99f-1b792222adbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:05.716044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b23eca-0d43-42c7-a99f-1b792222adbd None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:05.716329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b23eca-0d43-42c7-a99f-1b792222adbd None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:05.716605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b23eca-0d43-42c7-a99f-1b792222adbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:05.717826 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b23eca-0d43-42c7-a99f-1b792222adbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:05.719290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b23eca-0d43-42c7-a99f-1b792222adbd None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:05.720946 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 63/118] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:05.725652 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b23eca-0d43-42c7-a99f-1b792222adbd None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:05.836256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c94304-ae79-4e50-a78b-ebf5e505956e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:05.847137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b23eca-0d43-42c7-a99f-1b792222adbd None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:05.859897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:05.876186 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c205f-55cd-469b-be9f-eeff534ac87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:05.877295 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c205f-55cd-469b-be9f-eeff534ac87a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:05.877731 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c205f-55cd-469b-be9f-eeff534ac87a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:05.878143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c205f-55cd-469b-be9f-eeff534ac87a None admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:05.879867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c205f-55cd-469b-be9f-eeff534ac87a None admin] RBAC: Authorizing `identity:get_project(project_id=b3780a3b8fe742949173e4396724f2e0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:05.882387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c205f-55cd-469b-be9f-eeff534ac87a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:05.885152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 56/119] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:05 2022] GET /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:05.886415 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c205f-55cd-469b-be9f-eeff534ac87a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:05.894099 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073dda6f-0ad5-4b37-8e23-1cbc0ba54b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:05.894807 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dda6f-0ad5-4b37-8e23-1cbc0ba54b37 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:05.895162 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dda6f-0ad5-4b37-8e23-1cbc0ba54b37 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:05.895475 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dda6f-0ad5-4b37-8e23-1cbc0ba54b37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:05.896325 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073dda6f-0ad5-4b37-8e23-1cbc0ba54b37 None admin] RBAC: Adding query filter params (role.id=8a864244c21042b88accaa76b9117665, scope.project.id=b3780a3b8fe742949173e4396724f2e0, user.id=852f3d41db194fdcbd8c6e771be4b167) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:05.896685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073dda6f-0ad5-4b37-8e23-1cbc0ba54b37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:05.898193 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073dda6f-0ad5-4b37-8e23-1cbc0ba54b37 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:05.909056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 64/120] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:35:05 2022] GET /identity/v3/role_assignments?user.id=852f3d41db194fdcbd8c6e771be4b167&scope.project.id=b3780a3b8fe742949173e4396724f2e0&role.id=8a864244c21042b88accaa76b9117665 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:05.912040 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073dda6f-0ad5-4b37-8e23-1cbc0ba54b37 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:05.947648 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:06.015116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c205f-55cd-469b-be9f-eeff534ac87a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:06.058973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b9264-c6a7-4f60-b6a7-9245d374e62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:06.059244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073dda6f-0ad5-4b37-8e23-1cbc0ba54b37 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:06.060159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b9264-c6a7-4f60-b6a7-9245d374e62f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:06.060159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b9264-c6a7-4f60-b6a7-9245d374e62f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:06.060379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b9264-c6a7-4f60-b6a7-9245d374e62f None admin] PATH_INFO: `/v3/users/852f3d41db194fdcbd8c6e771be4b167` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:06.061521 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b9264-c6a7-4f60-b6a7-9245d374e62f None admin] RBAC: Authorizing `identity:get_user(user_id=852f3d41db194fdcbd8c6e771be4b167)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:06.063136 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b9264-c6a7-4f60-b6a7-9245d374e62f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:06.071920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 57/121] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:05 2022] GET /identity/v3/users/852f3d41db194fdcbd8c6e771be4b167 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:06.071920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b9264-c6a7-4f60-b6a7-9245d374e62f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:06.109146 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916360e0-7bb0-4f85-8ec5-7f9d215c48f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:06.110496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916360e0-7bb0-4f85-8ec5-7f9d215c48f3 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:06.110847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916360e0-7bb0-4f85-8ec5-7f9d215c48f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:06.112237 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916360e0-7bb0-4f85-8ec5-7f9d215c48f3 None admin] PATH_INFO: `/v3/users/137c28da9b8b4e96a2c74e35f43383d7` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:06.133185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916360e0-7bb0-4f85-8ec5-7f9d215c48f3 None admin] RBAC: Authorizing `identity:get_user(user_id=137c28da9b8b4e96a2c74e35f43383d7)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:06.135238 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916360e0-7bb0-4f85-8ec5-7f9d215c48f3 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:06.144255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 65/122] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:05 2022] GET /identity/v3/users/137c28da9b8b4e96a2c74e35f43383d7 => generated 289 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:06.144255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916360e0-7bb0-4f85-8ec5-7f9d215c48f3 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:06.186986 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:06.206004 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b9264-c6a7-4f60-b6a7-9245d374e62f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:06.227101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:06.257707 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4c0e8-9c87-4dcc-8de6-db4e6919824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:06.258631 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4c0e8-9c87-4dcc-8de6-db4e6919824c None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:06.259099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4c0e8-9c87-4dcc-8de6-db4e6919824c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:06.259541 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4c0e8-9c87-4dcc-8de6-db4e6919824c None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:06.260452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e4c0e8-9c87-4dcc-8de6-db4e6919824c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:06.262840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e4c0e8-9c87-4dcc-8de6-db4e6919824c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:06.293329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916360e0-7bb0-4f85-8ec5-7f9d215c48f3 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:06.301481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 58/123] 10.222.0.36 () {60 vars in 1141 bytes} [Sat Oct 15 13:35:06 2022] POST /identity/v3/projects => generated 302 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:35:06.308996 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4c0e8-9c87-4dcc-8de6-db4e6919824c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:06.339345 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8e5e0-0ee7-4d3c-a1b9-f90d1757311e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:06.340432 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8e5e0-0ee7-4d3c-a1b9-f90d1757311e None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:06.340921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8e5e0-0ee7-4d3c-a1b9-f90d1757311e None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:06.341425 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8e5e0-0ee7-4d3c-a1b9-f90d1757311e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:06.343123 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8e5e0-0ee7-4d3c-a1b9-f90d1757311e None admin] RBAC: Adding query filter params (role.id=ad5c2cf379e34fdcb85aa27f334e3fc0, scope.project.id=b3780a3b8fe742949173e4396724f2e0, user.id=852f3d41db194fdcbd8c6e771be4b167) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:06.343367 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8e5e0-0ee7-4d3c-a1b9-f90d1757311e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:06.347068 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8e5e0-0ee7-4d3c-a1b9-f90d1757311e None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:06.355488 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 66/124] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:35:06 2022] GET /identity/v3/role_assignments?user.id=852f3d41db194fdcbd8c6e771be4b167&scope.project.id=b3780a3b8fe742949173e4396724f2e0&role.id=ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:06.356810 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8e5e0-0ee7-4d3c-a1b9-f90d1757311e None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:06.500407 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4c0e8-9c87-4dcc-8de6-db4e6919824c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:06.558021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446c7fc-00b5-4d97-bc4b-42c4b8183cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:06.558694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446c7fc-00b5-4d97-bc4b-42c4b8183cb1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:06.558975 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446c7fc-00b5-4d97-bc4b-42c4b8183cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:06.559246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446c7fc-00b5-4d97-bc4b-42c4b8183cb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:06.571333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:06.581190 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8e5e0-0ee7-4d3c-a1b9-f90d1757311e None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:06.603727 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:06.629402 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dea52b-1091-46ee-9154-389e88020a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:06.630127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dea52b-1091-46ee-9154-389e88020a45 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:06.630408 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dea52b-1091-46ee-9154-389e88020a45 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:06.630682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dea52b-1091-46ee-9154-389e88020a45 None admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:06.632383 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dea52b-1091-46ee-9154-389e88020a45 None admin] RBAC: Authorizing `identity:get_project(project_id=b3780a3b8fe742949173e4396724f2e0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:06.633575 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dea52b-1091-46ee-9154-389e88020a45 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:06.635306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 67/125] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:06 2022] GET /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:06.641756 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dea52b-1091-46ee-9154-389e88020a45 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:06.655980 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446c7fc-00b5-4d97-bc4b-42c4b8183cb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:06.656660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446c7fc-00b5-4d97-bc4b-42c4b8183cb1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:06.779921 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 59/126] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:35:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:35:06.781009 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446c7fc-00b5-4d97-bc4b-42c4b8183cb1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:06.788644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dea52b-1091-46ee-9154-389e88020a45 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:06.825347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63243726-879d-46f9-bc1c-e455f2e4d8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:06.826019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243726-879d-46f9-bc1c-e455f2e4d8a0 None admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:06.826435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243726-879d-46f9-bc1c-e455f2e4d8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:06.826672 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243726-879d-46f9-bc1c-e455f2e4d8a0 None admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0/users/137c28da9b8b4e96a2c74e35f43383d7/roles/ba24faa1d7cb45f58bcb3bade9357c4c` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:06.828575 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63243726-879d-46f9-bc1c-e455f2e4d8a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3780a3b8fe742949173e4396724f2e0, user_id=137c28da9b8b4e96a2c74e35f43383d7, role_id=ba24faa1d7cb45f58bcb3bade9357c4c)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:06.830075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63243726-879d-46f9-bc1c-e455f2e4d8a0 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:06.845119 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 68/127] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:35:06 2022] PUT /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0/users/137c28da9b8b4e96a2c74e35f43383d7/roles/ba24faa1d7cb45f58bcb3bade9357c4c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:35:06.853569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:06.901952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446c7fc-00b5-4d97-bc4b-42c4b8183cb1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:06.940849 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7967860-d6b5-4d02-811a-ae20feec6326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:06.941511 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7967860-d6b5-4d02-811a-ae20feec6326 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:06.941821 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7967860-d6b5-4d02-811a-ae20feec6326 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:06.942093 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7967860-d6b5-4d02-811a-ae20feec6326 None admin] PATH_INFO: `/v3/projects/c3903475dd6549c19154391aacc9b814` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:06.943370 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7967860-d6b5-4d02-811a-ae20feec6326 None admin] RBAC: Authorizing `identity:get_project(project_id=c3903475dd6549c19154391aacc9b814)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:06.944956 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7967860-d6b5-4d02-811a-ae20feec6326 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:06.947058 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 60/128] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:06 2022] GET /identity/v3/projects/c3903475dd6549c19154391aacc9b814 => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:06.953281 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63243726-879d-46f9-bc1c-e455f2e4d8a0 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:06.984655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:07.106608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63243726-879d-46f9-bc1c-e455f2e4d8a0 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:07.160113 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9aa68-339b-4b97-978c-6b9502ca77fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:07.160997 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9aa68-339b-4b97-978c-6b9502ca77fc None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:07.161325 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9aa68-339b-4b97-978c-6b9502ca77fc None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:07.161667 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9aa68-339b-4b97-978c-6b9502ca77fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:07.162589 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9aa68-339b-4b97-978c-6b9502ca77fc None admin] RBAC: Adding query filter params (role.id=ad5c2cf379e34fdcb85aa27f334e3fc0, scope.project.id=c3903475dd6549c19154391aacc9b814, user.id=852f3d41db194fdcbd8c6e771be4b167) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:07.162971 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9aa68-339b-4b97-978c-6b9502ca77fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:07.164821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9aa68-339b-4b97-978c-6b9502ca77fc None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:07.179914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 69/129] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:35:06 2022] GET /identity/v3/role_assignments?user.id=852f3d41db194fdcbd8c6e771be4b167&scope.project.id=c3903475dd6549c19154391aacc9b814&role.id=ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:08.894864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4b3e2-6e61-4421-8792-375b3e78fe60 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:08.895418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4b3e2-6e61-4421-8792-375b3e78fe60 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:08.895830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4b3e2-6e61-4421-8792-375b3e78fe60 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:08.898434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 61/130] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:08.905773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b36aac-6104-49e0-872a-470dfb6c4658 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:08.906377 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b36aac-6104-49e0-872a-470dfb6c4658 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:08.906702 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b36aac-6104-49e0-872a-470dfb6c4658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:08.937718 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-70b36aac-6104-49e0-872a-470dfb6c4658 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:08.939817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-70b36aac-6104-49e0-872a-470dfb6c4658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:09.060546 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 70/131] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:09.262539 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5171b4e-70e7-4214-b40a-9e0389639e71 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:09.262790 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5171b4e-70e7-4214-b40a-9e0389639e71 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:09.263159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5171b4e-70e7-4214-b40a-9e0389639e71 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:09.264675 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 62/132] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:35:09.270567 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e5f9-3585-47f7-a968-02124e072412 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:09.270970 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e5f9-3585-47f7-a968-02124e072412 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:09.271316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e5f9-3585-47f7-a968-02124e072412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:09.290782 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-d259e5f9-3585-47f7-a968-02124e072412 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:09.292109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-d259e5f9-3585-47f7-a968-02124e072412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:09.307126 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:09.333217 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5171b4e-70e7-4214-b40a-9e0389639e71 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:09.373701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 71/133] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:09.459354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5171b4e-70e7-4214-b40a-9e0389639e71 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:09.498057 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768cf05-c0eb-47e5-bf6f-d5c8a0aaabbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:09.498815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768cf05-c0eb-47e5-bf6f-d5c8a0aaabbf None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:09.499203 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768cf05-c0eb-47e5-bf6f-d5c8a0aaabbf None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:09.499566 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768cf05-c0eb-47e5-bf6f-d5c8a0aaabbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:09.500840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768cf05-c0eb-47e5-bf6f-d5c8a0aaabbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:09.502545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768cf05-c0eb-47e5-bf6f-d5c8a0aaabbf None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:09.504286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 63/134] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:09.513945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259e5f9-3585-47f7-a968-02124e072412 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:09.630805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259e5f9-3585-47f7-a968-02124e072412 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:09.643527 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768cf05-c0eb-47e5-bf6f-d5c8a0aaabbf None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:09.647967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:09.674758 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2758de5-24f1-4b1c-bad9-bc72015efdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:09.675383 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2758de5-24f1-4b1c-bad9-bc72015efdca None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:09.675664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2758de5-24f1-4b1c-bad9-bc72015efdca None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:09.675968 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2758de5-24f1-4b1c-bad9-bc72015efdca None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:09.676643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2758de5-24f1-4b1c-bad9-bc72015efdca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:09.678109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2758de5-24f1-4b1c-bad9-bc72015efdca None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:09.702078 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 72/135] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:35:09 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:35:09.773095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768cf05-c0eb-47e5-bf6f-d5c8a0aaabbf None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:09.814671 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6cf90-072c-415c-aae3-61a88cc5bdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:09.815420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6cf90-072c-415c-aae3-61a88cc5bdc9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:09.815700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6cf90-072c-415c-aae3-61a88cc5bdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:09.816003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6cf90-072c-415c-aae3-61a88cc5bdc9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:09.828719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6cf90-072c-415c-aae3-61a88cc5bdc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-fcd6cf90-072c-415c-aae3-61a88cc5bdc9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:09.834316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:09.836368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:09.837932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:09.837932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:09.837932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:09.837932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:09.837932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:09.837932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:09.837932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:09.837932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:09.837932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:09.837932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:09.837932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 13:35:09.837932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:09.837932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 64/136] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:35:09.846786 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2758de5-24f1-4b1c-bad9-bc72015efdca None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:09.853393 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:09.998817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2758de5-24f1-4b1c-bad9-bc72015efdca None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:10.052109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9676e1f-d57b-4396-8236-9a93a0f70ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:10.052856 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9676e1f-d57b-4396-8236-9a93a0f70ef4 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:10.053263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9676e1f-d57b-4396-8236-9a93a0f70ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:10.053677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9676e1f-d57b-4396-8236-9a93a0f70ef4 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:10.054515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9676e1f-d57b-4396-8236-9a93a0f70ef4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:10.054948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9676e1f-d57b-4396-8236-9a93a0f70ef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:10.057567 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9676e1f-d57b-4396-8236-9a93a0f70ef4 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:10.070788 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 73/137] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:10.077790 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6cf90-072c-415c-aae3-61a88cc5bdc9 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:10.184495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:10.260672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6cf90-072c-415c-aae3-61a88cc5bdc9 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:10.330593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e3412-c3c5-455c-bcd3-1a8b765e912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:10.331581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e3412-c3c5-455c-bcd3-1a8b765e912c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:10.332113 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e3412-c3c5-455c-bcd3-1a8b765e912c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:10.332614 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e3412-c3c5-455c-bcd3-1a8b765e912c None admin] PATH_INFO: `/v3/users/137c28da9b8b4e96a2c74e35f43383d7` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:10.334635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e3412-c3c5-455c-bcd3-1a8b765e912c None admin] RBAC: Authorizing `identity:get_user(user_id=137c28da9b8b4e96a2c74e35f43383d7)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:10.337354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e3412-c3c5-455c-bcd3-1a8b765e912c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:10.340739 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 65/138] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:10 2022] GET /identity/v3/users/137c28da9b8b4e96a2c74e35f43383d7 => generated 289 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:10.350513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9676e1f-d57b-4396-8236-9a93a0f70ef4 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:10.438235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:10.483336 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9676e1f-d57b-4396-8236-9a93a0f70ef4 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:10.521616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169703e-58dc-4c29-abbf-3169f9c8db4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:10.522564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169703e-58dc-4c29-abbf-3169f9c8db4f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:10.522981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169703e-58dc-4c29-abbf-3169f9c8db4f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:10.523700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169703e-58dc-4c29-abbf-3169f9c8db4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:10.605223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169703e-58dc-4c29-abbf-3169f9c8db4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:10.606943 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169703e-58dc-4c29-abbf-3169f9c8db4f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:10.682832 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:10.721993 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 74/139] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:35:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:35:10.728215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e3412-c3c5-455c-bcd3-1a8b765e912c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:10.897262 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e3412-c3c5-455c-bcd3-1a8b765e912c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:10.952484 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:10.963176 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84798239-3189-488f-8227-d9b2745d845b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:10.964269 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84798239-3189-488f-8227-d9b2745d845b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:10.964745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84798239-3189-488f-8227-d9b2745d845b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:10.965202 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84798239-3189-488f-8227-d9b2745d845b None admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:10.967139 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84798239-3189-488f-8227-d9b2745d845b None admin] RBAC: Authorizing `identity:get_project(project_id=b3780a3b8fe742949173e4396724f2e0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:10.969770 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84798239-3189-488f-8227-d9b2745d845b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:10.973367 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 66/140] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:10 2022] GET /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:10.979396 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169703e-58dc-4c29-abbf-3169f9c8db4f None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:11.087339 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169703e-58dc-4c29-abbf-3169f9c8db4f None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:11.124655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75b54b-7ee3-4caa-91a1-63be11afcd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:11.125345 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75b54b-7ee3-4caa-91a1-63be11afcd22 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:11.125626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75b54b-7ee3-4caa-91a1-63be11afcd22 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:11.125898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75b54b-7ee3-4caa-91a1-63be11afcd22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:11.126640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75b54b-7ee3-4caa-91a1-63be11afcd22 None admin] RBAC: Adding query filter params (role.id=ba24faa1d7cb45f58bcb3bade9357c4c, scope.project.id=b3780a3b8fe742949173e4396724f2e0, user.id=137c28da9b8b4e96a2c74e35f43383d7) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:11.127003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75b54b-7ee3-4caa-91a1-63be11afcd22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:11.128627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75b54b-7ee3-4caa-91a1-63be11afcd22 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:11.135798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 75/141] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:35:10 2022] GET /identity/v3/role_assignments?user.id=137c28da9b8b4e96a2c74e35f43383d7&scope.project.id=b3780a3b8fe742949173e4396724f2e0&role.id=ba24faa1d7cb45f58bcb3bade9357c4c => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:12.486604 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52125502-4226-411f-ae9d-217172ee48f1 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:12.487659 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52125502-4226-411f-ae9d-217172ee48f1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:12.487659 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52125502-4226-411f-ae9d-217172ee48f1 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:12.488774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 67/142] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:12.494316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bb444-a660-4ac4-80d5-51df792065d4 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:12.495030 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bb444-a660-4ac4-80d5-51df792065d4 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:12.495030 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bb444-a660-4ac4-80d5-51df792065d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:12.514248 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-304bb444-a660-4ac4-80d5-51df792065d4 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:12.515456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-304bb444-a660-4ac4-80d5-51df792065d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:12.544553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:12.596752 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 76/143] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:12.670880 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b39a7-b7e8-49b8-bf42-ca0645bfdc47 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:12.671455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b39a7-b7e8-49b8-bf42-ca0645bfdc47 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:12.672067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b39a7-b7e8-49b8-bf42-ca0645bfdc47 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:12.674820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 68/144] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:12.681079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba27b58-d33a-494d-bdea-7633df3848b2 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:12.681342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba27b58-d33a-494d-bdea-7633df3848b2 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:12.681670 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba27b58-d33a-494d-bdea-7633df3848b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:12.708960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-dba27b58-d33a-494d-bdea-7633df3848b2 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:12.711028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-dba27b58-d33a-494d-bdea-7633df3848b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:12.805179 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 77/145] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:12.909448 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b39a7-b7e8-49b8-bf42-ca0645bfdc47 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:13.041597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b39a7-b7e8-49b8-bf42-ca0645bfdc47 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:13.081415 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba27b58-d33a-494d-bdea-7633df3848b2 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:13.092132 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45e8e5-f124-4dfb-8b21-4dfa59560abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:13.093172 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45e8e5-f124-4dfb-8b21-4dfa59560abd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:13.093617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45e8e5-f124-4dfb-8b21-4dfa59560abd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:13.094787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45e8e5-f124-4dfb-8b21-4dfa59560abd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:13.114698 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45e8e5-f124-4dfb-8b21-4dfa59560abd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-dd45e8e5-f124-4dfb-8b21-4dfa59560abd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:13.123279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:13.124274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:13.125041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:13.125041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:13.125041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:13.125041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:13.125041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:13.125041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:13.125041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:13.125041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:13.125041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:13.125041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:13.125041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:13.125041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:13.125832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 69/146] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:35:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:13.133228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45e8e5-f124-4dfb-8b21-4dfa59560abd None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:13.241302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:13.258255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:13.278601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba27b58-d33a-494d-bdea-7633df3848b2 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:13.308222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45e8e5-f124-4dfb-8b21-4dfa59560abd None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:13.330425 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90aec1-dce3-4762-850e-94f39bcb2cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:13.331271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90aec1-dce3-4762-850e-94f39bcb2cd7 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:13.331791 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90aec1-dce3-4762-850e-94f39bcb2cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:13.332184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90aec1-dce3-4762-850e-94f39bcb2cd7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:13.347900 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90aec1-dce3-4762-850e-94f39bcb2cd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:13.351020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a88ade-0e19-4408-8a45-da99821c1d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:13.352137 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a88ade-0e19-4408-8a45-da99821c1d8c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:13.352584 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a88ade-0e19-4408-8a45-da99821c1d8c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:13.353028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a88ade-0e19-4408-8a45-da99821c1d8c None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:13.354163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a88ade-0e19-4408-8a45-da99821c1d8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:13.354667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a88ade-0e19-4408-8a45-da99821c1d8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-5d90aec1-dce3-4762-850e-94f39bcb2cd7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:13.357410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:13.358316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:13.359091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:13.359091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:13.359091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:13.359091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:13.359091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:13.359091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:13.359091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:13.359091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:13.359091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:13.359091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:13.359091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:35:13.359091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:13.359683 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a88ade-0e19-4408-8a45-da99821c1d8c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:13.364279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 78/147] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:35:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:13.369402 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90aec1-dce3-4762-850e-94f39bcb2cd7 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:13.378726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 70/148] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:35:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:13.380428 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a88ade-0e19-4408-8a45-da99821c1d8c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:13.509557 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90aec1-dce3-4762-850e-94f39bcb2cd7 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:13.534136 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a88ade-0e19-4408-8a45-da99821c1d8c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:13.564215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:13.565207 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0bcc-7aff-4e3c-83c5-bcb5e03d187b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:13.565881 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0bcc-7aff-4e3c-83c5-bcb5e03d187b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:13.566164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0bcc-7aff-4e3c-83c5-bcb5e03d187b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:13.566450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b0bcc-7aff-4e3c-83c5-bcb5e03d187b None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:13.567306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b0bcc-7aff-4e3c-83c5-bcb5e03d187b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:13.568292 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b0bcc-7aff-4e3c-83c5-bcb5e03d187b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:13.569469 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b0bcc-7aff-4e3c-83c5-bcb5e03d187b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:13.582674 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 79/149] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:13.588497 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bca2f4-f702-48e8-97fb-164a73d980be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:13.589381 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0bcc-7aff-4e3c-83c5-bcb5e03d187b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:13.589605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bca2f4-f702-48e8-97fb-164a73d980be None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:13.590139 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bca2f4-f702-48e8-97fb-164a73d980be None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:13.590578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bca2f4-f702-48e8-97fb-164a73d980be None admin] PATH_INFO: `/v3/users/4e2cb51260c643abb333b8272d2c03c5` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:13.608440 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bca2f4-f702-48e8-97fb-164a73d980be None admin] RBAC: Authorizing `identity:get_user(user_id=4e2cb51260c643abb333b8272d2c03c5)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:13.610587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bca2f4-f702-48e8-97fb-164a73d980be None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:13.610939 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:13.612589 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 71/150] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:13 2022] GET /identity/v3/users/4e2cb51260c643abb333b8272d2c03c5 => generated 283 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:13.618542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bca2f4-f702-48e8-97fb-164a73d980be None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:13.729011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b0bcc-7aff-4e3c-83c5-bcb5e03d187b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:13.750270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bca2f4-f702-48e8-97fb-164a73d980be None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:13.769590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ac62b-d296-4ddb-910b-75e7ea7b176a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:13.770450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ac62b-d296-4ddb-910b-75e7ea7b176a None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:13.770807 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ac62b-d296-4ddb-910b-75e7ea7b176a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:13.771145 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ac62b-d296-4ddb-910b-75e7ea7b176a None admin] PATH_INFO: `/v3/users/4e2cb51260c643abb333b8272d2c03c5` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:13.772523 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ac62b-d296-4ddb-910b-75e7ea7b176a None admin] RBAC: Authorizing `identity:get_user(user_id=4e2cb51260c643abb333b8272d2c03c5)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:13.774284 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ac62b-d296-4ddb-910b-75e7ea7b176a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:13.776483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 80/151] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:13 2022] GET /identity/v3/users/4e2cb51260c643abb333b8272d2c03c5 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:13.777816 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b6a81-98a0-43a4-95b5-9f32f9f22c2b None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:13.778281 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b6a81-98a0-43a4-95b5-9f32f9f22c2b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:13.778727 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b6a81-98a0-43a4-95b5-9f32f9f22c2b None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:13.780641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 81/152] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:13.785578 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b6a81-98a0-43a4-95b5-9f32f9f22c2b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:13.800842 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd46f5-c6dc-48be-a933-f210f1b80482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:13.801891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd46f5-c6dc-48be-a933-f210f1b80482 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:13.802334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd46f5-c6dc-48be-a933-f210f1b80482 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:13.802704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd46f5-c6dc-48be-a933-f210f1b80482 None admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:13.804520 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd46f5-c6dc-48be-a933-f210f1b80482 None admin] RBAC: Authorizing `identity:get_project(project_id=34af9b28c3d641ddba8bde542506dc85)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:13.806797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd46f5-c6dc-48be-a933-f210f1b80482 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:13.809769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 72/153] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:13 2022] GET /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:13.811113 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845cc4d-eae8-47e1-a534-79ff301c84ec None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:13.811448 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845cc4d-eae8-47e1-a534-79ff301c84ec None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:13.811758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845cc4d-eae8-47e1-a534-79ff301c84ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:13.836718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2845cc4d-eae8-47e1-a534-79ff301c84ec None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:13.838524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2845cc4d-eae8-47e1-a534-79ff301c84ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:13.901596 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:13.923007 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:13.931687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 73/154] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:13.932956 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845cc4d-eae8-47e1-a534-79ff301c84ec None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:13.955411 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b6a81-98a0-43a4-95b5-9f32f9f22c2b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:14.016896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063c25a-829d-44c3-a800-658b38b4283e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:14.017891 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063c25a-829d-44c3-a800-658b38b4283e None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:14.018264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063c25a-829d-44c3-a800-658b38b4283e None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:14.018264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063c25a-829d-44c3-a800-658b38b4283e None admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:14.019402 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063c25a-829d-44c3-a800-658b38b4283e None admin] RBAC: Authorizing `identity:get_project(project_id=34af9b28c3d641ddba8bde542506dc85)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:14.020915 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063c25a-829d-44c3-a800-658b38b4283e None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:14.030388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 82/155] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:13 2022] GET /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:14.031491 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063c25a-829d-44c3-a800-658b38b4283e None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:14.111425 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845cc4d-eae8-47e1-a534-79ff301c84ec None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:14.170892 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f257a-1d3f-4f2c-a529-9868b9bca062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:14.171909 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f257a-1d3f-4f2c-a529-9868b9bca062 None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:14.172286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f257a-1d3f-4f2c-a529-9868b9bca062 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:14.172581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f257a-1d3f-4f2c-a529-9868b9bca062 None admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85/users/4e2cb51260c643abb333b8272d2c03c5/roles/8a864244c21042b88accaa76b9117665` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:14.176280 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f257a-1d3f-4f2c-a529-9868b9bca062 None admin] RBAC: Authorizing `identity:create_grant(project_id=34af9b28c3d641ddba8bde542506dc85, user_id=4e2cb51260c643abb333b8272d2c03c5, role_id=8a864244c21042b88accaa76b9117665)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:14.178518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f257a-1d3f-4f2c-a529-9868b9bca062 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:14.197245 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063c25a-829d-44c3-a800-658b38b4283e None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:14.201620 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 74/156] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:35:13 2022] PUT /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85/users/4e2cb51260c643abb333b8272d2c03c5/roles/8a864244c21042b88accaa76b9117665 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:35:14.203284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f257a-1d3f-4f2c-a529-9868b9bca062 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:14.247069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706eb6a8-142e-43dd-af41-2ab923dc8ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:14.248347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706eb6a8-142e-43dd-af41-2ab923dc8ac9 None admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:14.248728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706eb6a8-142e-43dd-af41-2ab923dc8ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:14.248980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706eb6a8-142e-43dd-af41-2ab923dc8ac9 None admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85/users/4e2cb51260c643abb333b8272d2c03c5/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:14.251173 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706eb6a8-142e-43dd-af41-2ab923dc8ac9 None admin] RBAC: Authorizing `identity:create_grant(project_id=34af9b28c3d641ddba8bde542506dc85, user_id=4e2cb51260c643abb333b8272d2c03c5, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:14.253084 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706eb6a8-142e-43dd-af41-2ab923dc8ac9 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:14.270020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:14.272988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 83/157] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:35:14 2022] PUT /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85/users/4e2cb51260c643abb333b8272d2c03c5/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:35:14.404729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f257a-1d3f-4f2c-a529-9868b9bca062 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:14.471508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb477aa9-d68e-4081-8c6c-7dd0012bedd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:14.472682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb477aa9-d68e-4081-8c6c-7dd0012bedd4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:14.473168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb477aa9-d68e-4081-8c6c-7dd0012bedd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:14.473659 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb477aa9-d68e-4081-8c6c-7dd0012bedd4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:14.475719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb477aa9-d68e-4081-8c6c-7dd0012bedd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:14.478467 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb477aa9-d68e-4081-8c6c-7dd0012bedd4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:14.482701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 75/158] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:14.490207 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706eb6a8-142e-43dd-af41-2ab923dc8ac9 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:14.510733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:14.619758 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706eb6a8-142e-43dd-af41-2ab923dc8ac9 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:14.658011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed76bc2-8380-4de4-96ef-ba38086f4684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:14.658882 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed76bc2-8380-4de4-96ef-ba38086f4684 None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:14.659135 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed76bc2-8380-4de4-96ef-ba38086f4684 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:14.659569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed76bc2-8380-4de4-96ef-ba38086f4684 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:14.660775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed76bc2-8380-4de4-96ef-ba38086f4684 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:14.663451 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed76bc2-8380-4de4-96ef-ba38086f4684 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:14.698745 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 84/159] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:35:14 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:35:17.133270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a7a0f-28f6-4c2e-92a2-6eb359c592b2 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:17.133843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a7a0f-28f6-4c2e-92a2-6eb359c592b2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:17.134200 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a7a0f-28f6-4c2e-92a2-6eb359c592b2 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:17.137125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 76/160] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:17.146858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65acd90-8be7-44a2-b9c6-437913b7a665 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:17.148372 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65acd90-8be7-44a2-b9c6-437913b7a665 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:17.149167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65acd90-8be7-44a2-b9c6-437913b7a665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:17.159196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011add4-60b2-4519-9b97-79d4614d46d2 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:17.159852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011add4-60b2-4519-9b97-79d4614d46d2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:17.160373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011add4-60b2-4519-9b97-79d4614d46d2 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:17.161976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 77/161] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:35:17.167803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8e29d-6156-4cea-9fdf-588fb8e2433d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:17.168447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8e29d-6156-4cea-9fdf-588fb8e2433d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:17.168920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8e29d-6156-4cea-9fdf-588fb8e2433d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:17.185545 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-e65acd90-8be7-44a2-b9c6-437913b7a665 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:17.188103 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e65acd90-8be7-44a2-b9c6-437913b7a665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:17.201390 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-57a8e29d-6156-4cea-9fdf-588fb8e2433d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:17.203681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-57a8e29d-6156-4cea-9fdf-588fb8e2433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:17.275842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:17.296466 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 85/162] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:17.332506 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 78/163] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:17.560675 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65acd90-8be7-44a2-b9c6-437913b7a665 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:17.612758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8e29d-6156-4cea-9fdf-588fb8e2433d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:17.669272 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:17.728881 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65acd90-8be7-44a2-b9c6-437913b7a665 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:17.775215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8e29d-6156-4cea-9fdf-588fb8e2433d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:17.791805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fdf349-bf4a-4438-b6cf-c16a8d2f4022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:17.792543 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdf349-bf4a-4438-b6cf-c16a8d2f4022 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:17.792823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdf349-bf4a-4438-b6cf-c16a8d2f4022 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:17.793091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdf349-bf4a-4438-b6cf-c16a8d2f4022 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:17.806464 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fdf349-bf4a-4438-b6cf-c16a8d2f4022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-d6fdf349-bf4a-4438-b6cf-c16a8d2f4022 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:17.816006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:17.817800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:17.819467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:17.819467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:17.819467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:17.819467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:17.819467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:17.819467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:17.819467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:17.819467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:17.819467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:17.819467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:17.819467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:35:17.819467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:17.824712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 86/164] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:35:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:17.827584 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c60f2-a295-430b-be0b-56df1dd48c7c None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:17.828198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c60f2-a295-430b-be0b-56df1dd48c7c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:17.828538 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c60f2-a295-430b-be0b-56df1dd48c7c None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:17.835935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 87/165] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:17.835935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c60f2-a295-430b-be0b-56df1dd48c7c None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:17.844306 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c159e6-c267-4904-a9d6-065d690f3faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:17.844973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c159e6-c267-4904-a9d6-065d690f3faa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:17.845247 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c159e6-c267-4904-a9d6-065d690f3faa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:17.845604 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c159e6-c267-4904-a9d6-065d690f3faa None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:17.860450 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c159e6-c267-4904-a9d6-065d690f3faa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-c9c159e6-c267-4904-a9d6-065d690f3faa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:17.866365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:17.869196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:17.870701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:17.870701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:17.870701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:17.870701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:17.870701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:17.870701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:17.870701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:17.870701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:17.870701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:17.870701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:17.870701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:17.870701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:17.870701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 79/166] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:35:17.870701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47fb36-34a1-4cb0-b781-71955a641631 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:17.872156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47fb36-34a1-4cb0-b781-71955a641631 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:17.872156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47fb36-34a1-4cb0-b781-71955a641631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:17.890201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7b47fb36-34a1-4cb0-b781-71955a641631 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:17.891717 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7b47fb36-34a1-4cb0-b781-71955a641631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:17.952746 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:17.976842 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 80/167] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:17.978044 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47fb36-34a1-4cb0-b781-71955a641631 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:18.016430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:18.045238 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c60f2-a295-430b-be0b-56df1dd48c7c None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:18.102656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad93f2-b391-4173-8cd8-0fa1452f4b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:18.104447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad93f2-b391-4173-8cd8-0fa1452f4b3b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:18.105250 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad93f2-b391-4173-8cd8-0fa1452f4b3b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:18.106082 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad93f2-b391-4173-8cd8-0fa1452f4b3b None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:18.106758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47fb36-34a1-4cb0-b781-71955a641631 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:18.107930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad93f2-b391-4173-8cd8-0fa1452f4b3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:18.109076 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad93f2-b391-4173-8cd8-0fa1452f4b3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:18.112657 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad93f2-b391-4173-8cd8-0fa1452f4b3b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:18.128620 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 88/168] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:18.140014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad93f2-b391-4173-8cd8-0fa1452f4b3b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:18.147175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d07c6-90af-4467-bbe2-b0e4f20e67e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:18.147828 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d07c6-90af-4467-bbe2-b0e4f20e67e9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:18.148113 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d07c6-90af-4467-bbe2-b0e4f20e67e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:18.148381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d07c6-90af-4467-bbe2-b0e4f20e67e9 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:18.149323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d07c6-90af-4467-bbe2-b0e4f20e67e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:18.149624 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d07c6-90af-4467-bbe2-b0e4f20e67e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:18.151177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d07c6-90af-4467-bbe2-b0e4f20e67e9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:18.166351 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 81/169] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:18.167795 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d07c6-90af-4467-bbe2-b0e4f20e67e9 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:18.271684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:18.277371 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad93f2-b391-4173-8cd8-0fa1452f4b3b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:18.290466 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d07c6-90af-4467-bbe2-b0e4f20e67e9 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:18.302503 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:18.328908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ad8c0-6a2b-4fb9-ad72-3076a82700f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:18.329695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ad8c0-6a2b-4fb9-ad72-3076a82700f3 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:18.330113 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ad8c0-6a2b-4fb9-ad72-3076a82700f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:18.330393 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ad8c0-6a2b-4fb9-ad72-3076a82700f3 None admin] PATH_INFO: `/v3/users/4e2cb51260c643abb333b8272d2c03c5` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:18.331745 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ad8c0-6a2b-4fb9-ad72-3076a82700f3 None admin] RBAC: Authorizing `identity:get_user(user_id=4e2cb51260c643abb333b8272d2c03c5)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:18.333291 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ad8c0-6a2b-4fb9-ad72-3076a82700f3 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:18.335143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 89/170] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:18 2022] GET /identity/v3/users/4e2cb51260c643abb333b8272d2c03c5 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:18.337187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c605f-976a-4f6b-90ae-2007f7aeddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:18.337635 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a426cc9-a49f-4440-a1fb-d353c7a2cd72 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:18.338007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c605f-976a-4f6b-90ae-2007f7aeddb0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:18.338321 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a426cc9-a49f-4440-a1fb-d353c7a2cd72 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:18.338420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c605f-976a-4f6b-90ae-2007f7aeddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:18.338785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c605f-976a-4f6b-90ae-2007f7aeddb0 None admin] PATH_INFO: `/v3/users/1ac71fedc08b4aadb00405afc09bce4e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:18.339085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a426cc9-a49f-4440-a1fb-d353c7a2cd72 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:18.340946 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 90/171] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:18.345496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a426cc9-a49f-4440-a1fb-d353c7a2cd72 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:18.363367 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c605f-976a-4f6b-90ae-2007f7aeddb0 None admin] RBAC: Authorizing `identity:get_user(user_id=1ac71fedc08b4aadb00405afc09bce4e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:18.364820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c605f-976a-4f6b-90ae-2007f7aeddb0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:18.366586 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 82/172] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:18 2022] GET /identity/v3/users/1ac71fedc08b4aadb00405afc09bce4e => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:18.367790 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8950f21-a6a5-4521-91c7-76b03b222efa None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:18.368094 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8950f21-a6a5-4521-91c7-76b03b222efa None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:18.368356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8950f21-a6a5-4521-91c7-76b03b222efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:18.389363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d8950f21-a6a5-4521-91c7-76b03b222efa None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:18.390568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d8950f21-a6a5-4521-91c7-76b03b222efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:18.480300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 83/173] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:18.480985 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8950f21-a6a5-4521-91c7-76b03b222efa None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:18.488347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a426cc9-a49f-4440-a1fb-d353c7a2cd72 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:18.546441 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a4f59-a2ec-4e41-9e4a-86f629b81ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:18.547062 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a4f59-a2ec-4e41-9e4a-86f629b81ba4 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:18.547339 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a4f59-a2ec-4e41-9e4a-86f629b81ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:18.549509 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a4f59-a2ec-4e41-9e4a-86f629b81ba4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:18.573626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:18.606502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8950f21-a6a5-4521-91c7-76b03b222efa None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:18.627024 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:18.650479 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b8521-9687-4551-a81e-a3b8f23205f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:18.651669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b8521-9687-4551-a81e-a3b8f23205f3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:18.651669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b8521-9687-4551-a81e-a3b8f23205f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:18.651964 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b8521-9687-4551-a81e-a3b8f23205f3 None admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:18.653003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b8521-9687-4551-a81e-a3b8f23205f3 None admin] RBAC: Authorizing `identity:get_project(project_id=34af9b28c3d641ddba8bde542506dc85)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:18.654509 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a4f59-a2ec-4e41-9e4a-86f629b81ba4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:18.654676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b8521-9687-4551-a81e-a3b8f23205f3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:18.663731 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 84/174] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:18 2022] GET /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:18.664149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a4f59-a2ec-4e41-9e4a-86f629b81ba4 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:18.664287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b8521-9687-4551-a81e-a3b8f23205f3 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:18.788647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b8521-9687-4551-a81e-a3b8f23205f3 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:18.817037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 91/175] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:35:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:35:18.817037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a4f59-a2ec-4e41-9e4a-86f629b81ba4 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:18.832881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379e5bb-6628-4431-9422-41d9f70175a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:18.833925 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379e5bb-6628-4431-9422-41d9f70175a3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:18.834411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379e5bb-6628-4431-9422-41d9f70175a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:18.834889 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379e5bb-6628-4431-9422-41d9f70175a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:18.852859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379e5bb-6628-4431-9422-41d9f70175a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-7379e5bb-6628-4431-9422-41d9f70175a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:18.862891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:18.863836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:18.864635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:18.864635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:18.864635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:18.864635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:18.864635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:18.864635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:18.864635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:18.864635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:18.864635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:18.864635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:18.864635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:18.864635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:18.865227 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 85/176] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:35:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:18.866256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379e5bb-6628-4431-9422-41d9f70175a3 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:18.871471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:18.931743 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:18.949697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a4f59-a2ec-4e41-9e4a-86f629b81ba4 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:18.986319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379e5bb-6628-4431-9422-41d9f70175a3 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:18.986969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe454d5-1a31-4414-8ec9-bcbee91e7fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:18.987671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe454d5-1a31-4414-8ec9-bcbee91e7fb2 None admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:18.987973 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe454d5-1a31-4414-8ec9-bcbee91e7fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:18.988241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe454d5-1a31-4414-8ec9-bcbee91e7fb2 None admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85/users/1ac71fedc08b4aadb00405afc09bce4e/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:18.990314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe454d5-1a31-4414-8ec9-bcbee91e7fb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=34af9b28c3d641ddba8bde542506dc85, user_id=1ac71fedc08b4aadb00405afc09bce4e, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:18.992260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe454d5-1a31-4414-8ec9-bcbee91e7fb2 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:19.008356 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 92/177] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:35:18 2022] PUT /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85/users/1ac71fedc08b4aadb00405afc09bce4e/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:35:19.009745 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe454d5-1a31-4414-8ec9-bcbee91e7fb2 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:19.037182 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4ae8e-db86-45c7-9c36-54f0e5e5147d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:19.038188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4ae8e-db86-45c7-9c36-54f0e5e5147d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:19.038461 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4ae8e-db86-45c7-9c36-54f0e5e5147d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:19.038726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4ae8e-db86-45c7-9c36-54f0e5e5147d None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:19.040118 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4ae8e-db86-45c7-9c36-54f0e5e5147d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:19.041662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4ae8e-db86-45c7-9c36-54f0e5e5147d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:19.043395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 86/178] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:19.049395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4ae8e-db86-45c7-9c36-54f0e5e5147d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:19.143280 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:19.161178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe454d5-1a31-4414-8ec9-bcbee91e7fb2 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:19.184278 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4ae8e-db86-45c7-9c36-54f0e5e5147d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:19.202404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31debbd-c476-4154-ad28-8740fa2603f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:19.203057 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31debbd-c476-4154-ad28-8740fa2603f7 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:19.203338 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31debbd-c476-4154-ad28-8740fa2603f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:19.203608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31debbd-c476-4154-ad28-8740fa2603f7 None admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:19.205192 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31debbd-c476-4154-ad28-8740fa2603f7 None admin] RBAC: Authorizing `identity:get_project(project_id=34af9b28c3d641ddba8bde542506dc85)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:19.206403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31debbd-c476-4154-ad28-8740fa2603f7 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:19.208181 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 93/179] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:19 2022] GET /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:19.210714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31debbd-c476-4154-ad28-8740fa2603f7 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:19.228320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:19.244466 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8b8b2-6079-468f-a586-74196ee1bce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:19.245226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8b8b2-6079-468f-a586-74196ee1bce3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:19.245504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8b8b2-6079-468f-a586-74196ee1bce3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:19.245771 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8b8b2-6079-468f-a586-74196ee1bce3 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:19.246439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8b8b2-6079-468f-a586-74196ee1bce3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:19.246743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8b8b2-6079-468f-a586-74196ee1bce3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:19.248169 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8b8b2-6079-468f-a586-74196ee1bce3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:19.257718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 87/180] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:35:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:19.264376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8b8b2-6079-468f-a586-74196ee1bce3 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:19.343978 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31debbd-c476-4154-ad28-8740fa2603f7 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:19.393826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8b8b2-6079-468f-a586-74196ee1bce3 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:19.398670 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a11640-aea9-4978-990a-9bf5c5b8b293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:19.400152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a11640-aea9-4978-990a-9bf5c5b8b293 None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:19.400862 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a11640-aea9-4978-990a-9bf5c5b8b293 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:19.401519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a11640-aea9-4978-990a-9bf5c5b8b293 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:19.402860 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a11640-aea9-4978-990a-9bf5c5b8b293 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:19.405897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a11640-aea9-4978-990a-9bf5c5b8b293 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:19.425594 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:19.431408 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1bc206-9a57-4328-94c6-0b1c4fb2fe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:19.432116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bc206-9a57-4328-94c6-0b1c4fb2fe9e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:19.432432 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bc206-9a57-4328-94c6-0b1c4fb2fe9e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:19.432645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bc206-9a57-4328-94c6-0b1c4fb2fe9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:19.433406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1bc206-9a57-4328-94c6-0b1c4fb2fe9e None admin] RBAC: Adding query filter params (role.id=57740dc2ebe1429294f20dd51592b82a, scope.project.id=34af9b28c3d641ddba8bde542506dc85, user.id=4e2cb51260c643abb333b8272d2c03c5) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:19.433953 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1bc206-9a57-4328-94c6-0b1c4fb2fe9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:19.435818 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1bc206-9a57-4328-94c6-0b1c4fb2fe9e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:19.447932 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 94/181] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:35:19 2022] POST /identity/v3/users => generated 297 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:35:19.447932 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a11640-aea9-4978-990a-9bf5c5b8b293 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:19.448348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 88/182] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:35:19 2022] GET /identity/v3/role_assignments?user.id=4e2cb51260c643abb333b8272d2c03c5&scope.project.id=34af9b28c3d641ddba8bde542506dc85&role.id=57740dc2ebe1429294f20dd51592b82a => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:19.575370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:19.653094 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a11640-aea9-4978-990a-9bf5c5b8b293 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:19.718766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26903c9d-5492-44bd-a51e-f731dae1a15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:19.719828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26903c9d-5492-44bd-a51e-f731dae1a15b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:19.720338 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26903c9d-5492-44bd-a51e-f731dae1a15b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:19.720887 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26903c9d-5492-44bd-a51e-f731dae1a15b None admin] PATH_INFO: `/v3/users/4e2cb51260c643abb333b8272d2c03c5` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:19.723236 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26903c9d-5492-44bd-a51e-f731dae1a15b None admin] RBAC: Authorizing `identity:get_user(user_id=4e2cb51260c643abb333b8272d2c03c5)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:19.725958 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26903c9d-5492-44bd-a51e-f731dae1a15b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:19.728977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 95/183] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:19 2022] GET /identity/v3/users/4e2cb51260c643abb333b8272d2c03c5 => generated 283 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:19.738354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1bc206-9a57-4328-94c6-0b1c4fb2fe9e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:19.867599 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1bc206-9a57-4328-94c6-0b1c4fb2fe9e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:19.924375 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1899e77-9e26-4953-a154-d292b948c924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:19.924924 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1899e77-9e26-4953-a154-d292b948c924 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:19.925202 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1899e77-9e26-4953-a154-d292b948c924 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:19.925474 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1899e77-9e26-4953-a154-d292b948c924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:20.003047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:20.011820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1899e77-9e26-4953-a154-d292b948c924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:20.013346 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1899e77-9e26-4953-a154-d292b948c924 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:20.137308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 89/184] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:35:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:35:20.144634 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26903c9d-5492-44bd-a51e-f731dae1a15b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:20.295433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26903c9d-5492-44bd-a51e-f731dae1a15b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:20.355671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304fa2e-1ff7-4863-ad69-66d4384c1b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:20.356674 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304fa2e-1ff7-4863-ad69-66d4384c1b9d None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:20.357080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304fa2e-1ff7-4863-ad69-66d4384c1b9d None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:20.357487 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304fa2e-1ff7-4863-ad69-66d4384c1b9d None admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:20.359382 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304fa2e-1ff7-4863-ad69-66d4384c1b9d None admin] RBAC: Authorizing `identity:get_project(project_id=34af9b28c3d641ddba8bde542506dc85)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:20.362291 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304fa2e-1ff7-4863-ad69-66d4384c1b9d None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:20.365496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 96/185] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:20 2022] GET /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:20.372162 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1899e77-9e26-4953-a154-d292b948c924 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:20.492107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:20.501122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1899e77-9e26-4953-a154-d292b948c924 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:20.548922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67467c24-97ce-4095-8ea5-8f880bf91e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:20.549465 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67467c24-97ce-4095-8ea5-8f880bf91e8f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:20.549740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67467c24-97ce-4095-8ea5-8f880bf91e8f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:20.550005 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67467c24-97ce-4095-8ea5-8f880bf91e8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:20.550772 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67467c24-97ce-4095-8ea5-8f880bf91e8f None admin] RBAC: Adding query filter params (role.id=8a864244c21042b88accaa76b9117665, scope.project.id=34af9b28c3d641ddba8bde542506dc85, user.id=4e2cb51260c643abb333b8272d2c03c5) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:20.551081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67467c24-97ce-4095-8ea5-8f880bf91e8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:20.552667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67467c24-97ce-4095-8ea5-8f880bf91e8f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:20.560159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 90/186] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:35:20 2022] GET /identity/v3/role_assignments?user.id=4e2cb51260c643abb333b8272d2c03c5&scope.project.id=34af9b28c3d641ddba8bde542506dc85&role.id=8a864244c21042b88accaa76b9117665 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:21.422579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b543f-a3ca-476b-a472-ee9b676685c0 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:21.423369 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b543f-a3ca-476b-a472-ee9b676685c0 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:21.423369 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b543f-a3ca-476b-a472-ee9b676685c0 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:21.424873 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 97/187] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:35:21.431414 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b5556-3569-4c85-92ba-b595ed1d8fe1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:21.432187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b5556-3569-4c85-92ba-b595ed1d8fe1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:21.432187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b5556-3569-4c85-92ba-b595ed1d8fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:21.454958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-727b5556-3569-4c85-92ba-b595ed1d8fe1 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:21.456402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-727b5556-3569-4c85-92ba-b595ed1d8fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:21.534299 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 91/188] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:21.804079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b543f-a3ca-476b-a472-ee9b676685c0 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:21.808457 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:21.976134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf3e2f-15d7-4e37-82d7-78391dab8e17 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:21.976393 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf3e2f-15d7-4e37-82d7-78391dab8e17 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:21.976771 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf3e2f-15d7-4e37-82d7-78391dab8e17 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:21.978063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 92/189] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:35:21.983699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b176659-234c-4d04-83b3-1fc704781576 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:21.984021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b176659-234c-4d04-83b3-1fc704781576 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:21.984293 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b176659-234c-4d04-83b3-1fc704781576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:22.001069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b543f-a3ca-476b-a472-ee9b676685c0 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:22.004717 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9b176659-234c-4d04-83b3-1fc704781576 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:22.005962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9b176659-234c-4d04-83b3-1fc704781576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:22.066916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf512ee-43c5-4dce-8e5c-c2dab667dfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:22.068411 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf512ee-43c5-4dce-8e5c-c2dab667dfb8 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:22.068991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf512ee-43c5-4dce-8e5c-c2dab667dfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:22.069494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf512ee-43c5-4dce-8e5c-c2dab667dfb8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:22.088654 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 93/190] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:22.096246 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf512ee-43c5-4dce-8e5c-c2dab667dfb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-6cf512ee-43c5-4dce-8e5c-c2dab667dfb8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:22.100423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:22.101959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:22.103355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:22.103355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:22.103355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:22.103355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:22.103355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:22.103355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:22.103355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:22.103355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:22.103355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:22.103355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:22.103355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:22.103355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:22.103355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 98/191] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:22.113744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b176659-234c-4d04-83b3-1fc704781576 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:22.154527 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:22.244923 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b27af8-7248-4ab1-95fb-53aec55276f8 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:22.245363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b27af8-7248-4ab1-95fb-53aec55276f8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:22.245823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b27af8-7248-4ab1-95fb-53aec55276f8 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:22.248312 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 99/192] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:35:22.254423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d006d1d-a05e-490d-8bb0-ad030a7b3b55 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:22.254753 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d006d1d-a05e-490d-8bb0-ad030a7b3b55 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:22.255030 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d006d1d-a05e-490d-8bb0-ad030a7b3b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:22.276702 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-5d006d1d-a05e-490d-8bb0-ad030a7b3b55 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:22.277969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-5d006d1d-a05e-490d-8bb0-ad030a7b3b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:22.303494 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b176659-234c-4d04-83b3-1fc704781576 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:22.348643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:22.360856 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 100/193] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:22.362385 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d006d1d-a05e-490d-8bb0-ad030a7b3b55 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:22.370002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2f156-213f-4f3f-a31b-ca1327235fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:22.371015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2f156-213f-4f3f-a31b-ca1327235fa2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:22.371473 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2f156-213f-4f3f-a31b-ca1327235fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:22.371969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2f156-213f-4f3f-a31b-ca1327235fa2 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:22.373098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2f156-213f-4f3f-a31b-ca1327235fa2 None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:22.373670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2f156-213f-4f3f-a31b-ca1327235fa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:22.376189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2f156-213f-4f3f-a31b-ca1327235fa2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:22.392383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 94/194] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:22.401383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2f156-213f-4f3f-a31b-ca1327235fa2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:22.534489 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d006d1d-a05e-490d-8bb0-ad030a7b3b55 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:22.544725 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:22.578409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2f156-213f-4f3f-a31b-ca1327235fa2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:22.580458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc161f2-67d4-4397-a6e4-c2f6d308d4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:22.581433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc161f2-67d4-4397-a6e4-c2f6d308d4ea None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:22.581792 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc161f2-67d4-4397-a6e4-c2f6d308d4ea None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:22.582223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc161f2-67d4-4397-a6e4-c2f6d308d4ea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:22.595483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc161f2-67d4-4397-a6e4-c2f6d308d4ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-3dc161f2-67d4-4397-a6e4-c2f6d308d4ea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:22.601644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:22.603350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:22.604344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:22.604344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:22.604344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:22.604344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:22.604344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:22.604344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:22.604344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:22.604344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:22.604344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:22.604344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:22.604344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 13:35:22.604344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:22.604344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 101/195] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:22.612937 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc161f2-67d4-4397-a6e4-c2f6d308d4ea None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:22.641809 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85be516-321e-44df-a545-299d9b3aca23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:22.642784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85be516-321e-44df-a545-299d9b3aca23 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:22.643190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85be516-321e-44df-a545-299d9b3aca23 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:22.643584 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85be516-321e-44df-a545-299d9b3aca23 None admin] PATH_INFO: `/v3/users/1ac71fedc08b4aadb00405afc09bce4e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:22.645177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85be516-321e-44df-a545-299d9b3aca23 None admin] RBAC: Authorizing `identity:get_user(user_id=1ac71fedc08b4aadb00405afc09bce4e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:22.646841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85be516-321e-44df-a545-299d9b3aca23 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:22.648874 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 95/196] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:22 2022] GET /identity/v3/users/1ac71fedc08b4aadb00405afc09bce4e => generated 297 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:22.655826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85be516-321e-44df-a545-299d9b3aca23 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:22.709227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:22.765711 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc161f2-67d4-4397-a6e4-c2f6d308d4ea None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:22.808777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85be516-321e-44df-a545-299d9b3aca23 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:22.812301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe9134-b0df-41c4-9bae-6847cbf3b971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:22.813220 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe9134-b0df-41c4-9bae-6847cbf3b971 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:22.813545 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe9134-b0df-41c4-9bae-6847cbf3b971 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:22.813865 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe9134-b0df-41c4-9bae-6847cbf3b971 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:22.814812 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe9134-b0df-41c4-9bae-6847cbf3b971 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:22.815161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe9134-b0df-41c4-9bae-6847cbf3b971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:22.816665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe9134-b0df-41c4-9bae-6847cbf3b971 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:22.826884 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 102/197] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:22.827756 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe9134-b0df-41c4-9bae-6847cbf3b971 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:22.857310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45ea5d-ed9c-4607-bcc8-a0a97174703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:22.858045 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45ea5d-ed9c-4607-bcc8-a0a97174703d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:22.858408 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45ea5d-ed9c-4607-bcc8-a0a97174703d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:22.858821 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45ea5d-ed9c-4607-bcc8-a0a97174703d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:22.885524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:22.955260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45ea5d-ed9c-4607-bcc8-a0a97174703d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:22.956892 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45ea5d-ed9c-4607-bcc8-a0a97174703d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:22.977764 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe9134-b0df-41c4-9bae-6847cbf3b971 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:23.005765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:23.032561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454448d2-22cc-4af5-85db-1afb1f5f9cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:23.033583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454448d2-22cc-4af5-85db-1afb1f5f9cdd None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:23.034034 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454448d2-22cc-4af5-85db-1afb1f5f9cdd None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:23.034472 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454448d2-22cc-4af5-85db-1afb1f5f9cdd None admin] PATH_INFO: `/v3/domains/default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:23.036256 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454448d2-22cc-4af5-85db-1afb1f5f9cdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:23.038476 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454448d2-22cc-4af5-85db-1afb1f5f9cdd None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:23.046465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 103/198] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:23.046465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454448d2-22cc-4af5-85db-1afb1f5f9cdd None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:23.084928 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 96/199] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:35:23.085960 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45ea5d-ed9c-4607-bcc8-a0a97174703d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:23.162720 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:23.172905 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454448d2-22cc-4af5-85db-1afb1f5f9cdd None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:23.210533 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a8c8e-d64f-498b-bfe6-df06269f477d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:23.211341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a8c8e-d64f-498b-bfe6-df06269f477d None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:23.211680 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a8c8e-d64f-498b-bfe6-df06269f477d None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:23.212112 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a8c8e-d64f-498b-bfe6-df06269f477d None admin] PATH_INFO: `/v3/users/e0253265e4c245c69757830e1a85e998` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:23.228024 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a8c8e-d64f-498b-bfe6-df06269f477d None admin] RBAC: Authorizing `identity:get_user(user_id=e0253265e4c245c69757830e1a85e998)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:23.229512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a8c8e-d64f-498b-bfe6-df06269f477d None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:23.231238 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 104/200] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:23 2022] GET /identity/v3/users/e0253265e4c245c69757830e1a85e998 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:23.232067 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a8c8e-d64f-498b-bfe6-df06269f477d None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:23.235014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45ea5d-ed9c-4607-bcc8-a0a97174703d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:23.276721 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16062d4a-839b-4682-b829-a812dc078a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:23.277492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16062d4a-839b-4682-b829-a812dc078a08 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:23.277875 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16062d4a-839b-4682-b829-a812dc078a08 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:23.278331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16062d4a-839b-4682-b829-a812dc078a08 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:23.279054 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16062d4a-839b-4682-b829-a812dc078a08 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:23.280859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16062d4a-839b-4682-b829-a812dc078a08 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:23.300451 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:23.306682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 97/201] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:35:23 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:35:23.308168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16062d4a-839b-4682-b829-a812dc078a08 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:23.397641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a8c8e-d64f-498b-bfe6-df06269f477d None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:23.435812 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16062d4a-839b-4682-b829-a812dc078a08 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:23.445061 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:23.446565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b313f-9995-4fdc-ba3f-f173e8816305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:23.447685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b313f-9995-4fdc-ba3f-f173e8816305 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:23.448205 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b313f-9995-4fdc-ba3f-f173e8816305 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:23.448680 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b313f-9995-4fdc-ba3f-f173e8816305 None admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:23.450840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b313f-9995-4fdc-ba3f-f173e8816305 None admin] RBAC: Authorizing `identity:get_project(project_id=34af9b28c3d641ddba8bde542506dc85)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:23.453237 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b313f-9995-4fdc-ba3f-f173e8816305 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:23.462012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 105/202] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:23 2022] GET /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:23.463230 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b313f-9995-4fdc-ba3f-f173e8816305 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:23.484016 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f30215-64d4-45d1-8290-c41b809944a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:23.485036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f30215-64d4-45d1-8290-c41b809944a4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:23.485319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f30215-64d4-45d1-8290-c41b809944a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:23.485582 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f30215-64d4-45d1-8290-c41b809944a4 None admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:23.486978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f30215-64d4-45d1-8290-c41b809944a4 None admin] RBAC: Authorizing `identity:get_project(project_id=34af9b28c3d641ddba8bde542506dc85)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:23.488435 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f30215-64d4-45d1-8290-c41b809944a4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:23.490465 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 98/203] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:23 2022] GET /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:23.496638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f30215-64d4-45d1-8290-c41b809944a4 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:23.615580 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b313f-9995-4fdc-ba3f-f173e8816305 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:23.634725 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:23.655836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f30215-64d4-45d1-8290-c41b809944a4 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:23.677915 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7354e-5b99-4f61-94a0-ac5d78761725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:23.678802 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7354e-5b99-4f61-94a0-ac5d78761725 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:23.679265 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7354e-5b99-4f61-94a0-ac5d78761725 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:23.679724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7354e-5b99-4f61-94a0-ac5d78761725 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:23.681035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7354e-5b99-4f61-94a0-ac5d78761725 None admin] RBAC: Adding query filter params (role.id=ad5c2cf379e34fdcb85aa27f334e3fc0, scope.project.id=34af9b28c3d641ddba8bde542506dc85, user.id=1ac71fedc08b4aadb00405afc09bce4e) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:23.681561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7354e-5b99-4f61-94a0-ac5d78761725 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:23.683276 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7354e-5b99-4f61-94a0-ac5d78761725 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:23.692709 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 106/204] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:35:23 2022] GET /identity/v3/role_assignments?user.id=1ac71fedc08b4aadb00405afc09bce4e&scope.project.id=34af9b28c3d641ddba8bde542506dc85&role.id=ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:23.710700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c297f-6e45-4f5f-b14b-ecf0d8b3426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:23.712200 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c297f-6e45-4f5f-b14b-ecf0d8b3426c None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:23.712627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c297f-6e45-4f5f-b14b-ecf0d8b3426c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:23.713116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c297f-6e45-4f5f-b14b-ecf0d8b3426c None admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85/users/e0253265e4c245c69757830e1a85e998/roles/ba24faa1d7cb45f58bcb3bade9357c4c` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:23.718087 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c297f-6e45-4f5f-b14b-ecf0d8b3426c None admin] RBAC: Authorizing `identity:create_grant(project_id=34af9b28c3d641ddba8bde542506dc85, user_id=e0253265e4c245c69757830e1a85e998, role_id=ba24faa1d7cb45f58bcb3bade9357c4c)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:23.721067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c297f-6e45-4f5f-b14b-ecf0d8b3426c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:23.746175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 99/205] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:35:23 2022] PUT /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85/users/e0253265e4c245c69757830e1a85e998/roles/ba24faa1d7cb45f58bcb3bade9357c4c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:35:25.910957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614bdf5-863e-437e-af4b-a4b38301b696 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:25.911823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614bdf5-863e-437e-af4b-a4b38301b696 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:25.911823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614bdf5-863e-437e-af4b-a4b38301b696 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:25.913477 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 107/206] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:35:25.920805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dda3b-3dba-4805-a4b2-a8ecacf60eb1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:25.921418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dda3b-3dba-4805-a4b2-a8ecacf60eb1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:25.921738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dda3b-3dba-4805-a4b2-a8ecacf60eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:25.954235 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e92dda3b-3dba-4805-a4b2-a8ecacf60eb1 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:25.956313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e92dda3b-3dba-4805-a4b2-a8ecacf60eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:26.000980 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:26.025640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f82ca9-0fe6-4952-ab51-1c9f725aae83 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:26.025933 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f82ca9-0fe6-4952-ab51-1c9f725aae83 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:26.026901 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f82ca9-0fe6-4952-ab51-1c9f725aae83 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:26.028221 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 108/207] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:35:26.034638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb9457-f913-4f1d-add9-ffd2ccf9bfe1 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:26.034902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb9457-f913-4f1d-add9-ffd2ccf9bfe1 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:26.035268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb9457-f913-4f1d-add9-ffd2ccf9bfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:26.055523 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-70cb9457-f913-4f1d-add9-ffd2ccf9bfe1 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:26.056812 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-70cb9457-f913-4f1d-add9-ffd2ccf9bfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:26.061084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 100/208] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:26.083726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3789076-e652-416d-8127-518398115220 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:26.084222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3789076-e652-416d-8127-518398115220 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:26.084696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3789076-e652-416d-8127-518398115220 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:26.088159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 101/209] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:26.094052 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461172a8-8442-45dd-9731-f18eb9d736be None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:26.094760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461172a8-8442-45dd-9731-f18eb9d736be None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:26.096736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461172a8-8442-45dd-9731-f18eb9d736be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:26.126995 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-461172a8-8442-45dd-9731-f18eb9d736be None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:26.129374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-461172a8-8442-45dd-9731-f18eb9d736be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:26.172942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 109/210] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:26.253653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 102/211] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:26.329431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb9457-f913-4f1d-add9-ffd2ccf9bfe1 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:26.457866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb9457-f913-4f1d-add9-ffd2ccf9bfe1 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:26.460933 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:26.479350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461172a8-8442-45dd-9731-f18eb9d736be None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:26.503994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fae74e-654c-416b-bd74-228e398e87dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:26.504667 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fae74e-654c-416b-bd74-228e398e87dd None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:26.504943 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fae74e-654c-416b-bd74-228e398e87dd None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:26.505214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fae74e-654c-416b-bd74-228e398e87dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:26.506431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fae74e-654c-416b-bd74-228e398e87dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:26.507965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fae74e-654c-416b-bd74-228e398e87dd None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:26.509668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 110/212] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:26.515847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fae74e-654c-416b-bd74-228e398e87dd None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:26.610641 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:26.622995 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461172a8-8442-45dd-9731-f18eb9d736be None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:26.648987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fae74e-654c-416b-bd74-228e398e87dd None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:26.675126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b7452-cda9-413e-b60f-6a3e5c72a9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:26.676731 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b7452-cda9-413e-b60f-6a3e5c72a9b8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:26.677455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b7452-cda9-413e-b60f-6a3e5c72a9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:26.678153 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b7452-cda9-413e-b60f-6a3e5c72a9b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:26.692693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b7452-cda9-413e-b60f-6a3e5c72a9b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-ba3b7452-cda9-413e-b60f-6a3e5c72a9b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:26.697787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:26.698872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:26.701298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:26.701298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:26.701298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:26.701298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:26.701298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:26.701298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:26.701298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:26.701298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:26.701298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:26.701298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:26.701298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 13:35:26.701298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:26.701298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 103/213] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:35:26.703540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dc437-f453-429e-8c70-94a0038eab17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:26.704218 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dc437-f453-429e-8c70-94a0038eab17 None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:26.704510 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dc437-f453-429e-8c70-94a0038eab17 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:26.704927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dc437-f453-429e-8c70-94a0038eab17 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:26.705891 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dc437-f453-429e-8c70-94a0038eab17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:26.707406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b7452-cda9-413e-b60f-6a3e5c72a9b8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:26.707586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dc437-f453-429e-8c70-94a0038eab17 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:26.735744 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 111/214] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:35:26 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:35:26.737360 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dc437-f453-429e-8c70-94a0038eab17 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:26.789016 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:26.886338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b7452-cda9-413e-b60f-6a3e5c72a9b8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:26.930301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dc437-f453-429e-8c70-94a0038eab17 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:26.934493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c8041-dd8a-4f51-a48a-68392d961c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:26.935286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c8041-dd8a-4f51-a48a-68392d961c84 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:26.935560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c8041-dd8a-4f51-a48a-68392d961c84 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:26.935866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c8041-dd8a-4f51-a48a-68392d961c84 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:26.936757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c8041-dd8a-4f51-a48a-68392d961c84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:26.937064 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c8041-dd8a-4f51-a48a-68392d961c84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:26.938534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c8041-dd8a-4f51-a48a-68392d961c84 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:26.949856 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 104/215] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:26.955939 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c8041-dd8a-4f51-a48a-68392d961c84 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:26.979227 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:26.998587 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c5179-b8f1-4636-b106-3b62ec363771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:26.999582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c5179-b8f1-4636-b106-3b62ec363771 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:26.999996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c5179-b8f1-4636-b106-3b62ec363771 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:27.000391 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c5179-b8f1-4636-b106-3b62ec363771 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:27.016852 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c5179-b8f1-4636-b106-3b62ec363771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-414c5179-b8f1-4636-b106-3b62ec363771 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:27.022747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:27.024650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:27.026196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:27.026196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:27.026196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:27.026196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:27.026196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:27.026196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:27.026196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:27.026196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:27.026196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:27.026196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:27.026196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:27.026196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:27.026196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 112/216] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:27.032162 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c5179-b8f1-4636-b106-3b62ec363771 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:27.131250 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c8041-dd8a-4f51-a48a-68392d961c84 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:27.178298 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:27.193122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9d262-8961-43c5-a8a4-8a3ec06da907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:27.194170 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9d262-8961-43c5-a8a4-8a3ec06da907 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:27.195265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9d262-8961-43c5-a8a4-8a3ec06da907 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:27.195969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9d262-8961-43c5-a8a4-8a3ec06da907 None admin] PATH_INFO: `/v3/users/e0253265e4c245c69757830e1a85e998` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:27.197879 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c5179-b8f1-4636-b106-3b62ec363771 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:27.198078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9d262-8961-43c5-a8a4-8a3ec06da907 None admin] RBAC: Authorizing `identity:get_user(user_id=e0253265e4c245c69757830e1a85e998)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:27.201629 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9d262-8961-43c5-a8a4-8a3ec06da907 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:27.205445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 105/217] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:26 2022] GET /identity/v3/users/e0253265e4c245c69757830e1a85e998 => generated 297 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:27.212772 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9d262-8961-43c5-a8a4-8a3ec06da907 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:27.247901 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947da0fa-ebc7-4156-9a0f-1727c0a43e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:27.248681 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947da0fa-ebc7-4156-9a0f-1727c0a43e5c None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:27.248867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947da0fa-ebc7-4156-9a0f-1727c0a43e5c None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:27.249151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947da0fa-ebc7-4156-9a0f-1727c0a43e5c None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:27.249889 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947da0fa-ebc7-4156-9a0f-1727c0a43e5c None admin] RBAC: Adding query filter params (name=member) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:27.250196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947da0fa-ebc7-4156-9a0f-1727c0a43e5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:27.251695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947da0fa-ebc7-4156-9a0f-1727c0a43e5c None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:27.264172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 113/218] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:27.268617 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947da0fa-ebc7-4156-9a0f-1727c0a43e5c None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:27.414346 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9d262-8961-43c5-a8a4-8a3ec06da907 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:27.420310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:27.462526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947da0fa-ebc7-4156-9a0f-1727c0a43e5c None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:27.484443 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba95a14-999b-42c5-933d-c825e2b0b502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:27.486149 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba95a14-999b-42c5-933d-c825e2b0b502 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:27.486868 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba95a14-999b-42c5-933d-c825e2b0b502 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:27.487548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba95a14-999b-42c5-933d-c825e2b0b502 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:27.526835 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c2d61-f275-4624-8388-93d34bfd3f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:27.528085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c2d61-f275-4624-8388-93d34bfd3f22 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:27.528727 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c2d61-f275-4624-8388-93d34bfd3f22 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:27.529329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c2d61-f275-4624-8388-93d34bfd3f22 None admin] PATH_INFO: `/v3/users/8ff1b1beacd84db3948ade79a5042be0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:27.555269 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c2d61-f275-4624-8388-93d34bfd3f22 None admin] RBAC: Authorizing `identity:get_user(user_id=8ff1b1beacd84db3948ade79a5042be0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:27.558263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c2d61-f275-4624-8388-93d34bfd3f22 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:27.561604 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 114/219] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:27 2022] GET /identity/v3/users/8ff1b1beacd84db3948ade79a5042be0 => generated 293 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:27.572101 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c2d61-f275-4624-8388-93d34bfd3f22 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:27.593453 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:27.625611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba95a14-999b-42c5-933d-c825e2b0b502 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:27.628841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba95a14-999b-42c5-933d-c825e2b0b502 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:27.761705 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c2d61-f275-4624-8388-93d34bfd3f22 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:27.780756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:27.813738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 106/220] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:35:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:35:27.822359 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba95a14-999b-42c5-933d-c825e2b0b502 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:27.824712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447ae82-ae05-48ec-801d-2e656e328551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:27.825819 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447ae82-ae05-48ec-801d-2e656e328551 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:27.826359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447ae82-ae05-48ec-801d-2e656e328551 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:27.826877 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447ae82-ae05-48ec-801d-2e656e328551 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:27.828268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447ae82-ae05-48ec-801d-2e656e328551 None admin] RBAC: Adding query filter params (role.id=ad5c2cf379e34fdcb85aa27f334e3fc0, user.id=8ff1b1beacd84db3948ade79a5042be0, scope.system=all) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:27.828862 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447ae82-ae05-48ec-801d-2e656e328551 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:27.831602 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447ae82-ae05-48ec-801d-2e656e328551 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:27.841800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 115/221] 10.222.0.36 () {56 vars in 1308 bytes} [Sat Oct 15 13:35:27 2022] GET /identity/v3/role_assignments?user.id=8ff1b1beacd84db3948ade79a5042be0&scope.system=all&role.id=ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:28.002649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba95a14-999b-42c5-933d-c825e2b0b502 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:28.069649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21084c-e8e6-4320-a693-21424e8aebb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:28.070687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21084c-e8e6-4320-a693-21424e8aebb0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:28.071145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21084c-e8e6-4320-a693-21424e8aebb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:28.071598 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21084c-e8e6-4320-a693-21424e8aebb0 None admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:28.073874 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21084c-e8e6-4320-a693-21424e8aebb0 None admin] RBAC: Authorizing `identity:get_project(project_id=34af9b28c3d641ddba8bde542506dc85)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:28.076711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21084c-e8e6-4320-a693-21424e8aebb0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:28.080131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 107/222] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:27 2022] GET /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:28.086774 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447ae82-ae05-48ec-801d-2e656e328551 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:28.177124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:28.204321 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447ae82-ae05-48ec-801d-2e656e328551 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:28.242993 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf796b7c-e77c-4cc8-848b-c55d06f90bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:28.243611 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf796b7c-e77c-4cc8-848b-c55d06f90bff None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:28.243909 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf796b7c-e77c-4cc8-848b-c55d06f90bff None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:28.244228 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf796b7c-e77c-4cc8-848b-c55d06f90bff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:28.244923 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf796b7c-e77c-4cc8-848b-c55d06f90bff None admin] RBAC: Adding query filter params (role.id=ba24faa1d7cb45f58bcb3bade9357c4c, scope.project.id=34af9b28c3d641ddba8bde542506dc85, user.id=e0253265e4c245c69757830e1a85e998) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:28.245229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf796b7c-e77c-4cc8-848b-c55d06f90bff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:28.246686 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf796b7c-e77c-4cc8-848b-c55d06f90bff None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:28.254003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 116/223] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:35:28 2022] GET /identity/v3/role_assignments?user.id=e0253265e4c245c69757830e1a85e998&scope.project.id=34af9b28c3d641ddba8bde542506dc85&role.id=ba24faa1d7cb45f58bcb3bade9357c4c => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:29.855967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b9b4d-92ee-4b26-aa77-724502201a8a None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:29.856537 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b9b4d-92ee-4b26-aa77-724502201a8a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:29.856909 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b9b4d-92ee-4b26-aa77-724502201a8a None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:29.859937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 108/224] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:29.868881 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108bbbe-b5c5-472a-b187-bac08b560940 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:29.869392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108bbbe-b5c5-472a-b187-bac08b560940 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:29.872762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108bbbe-b5c5-472a-b187-bac08b560940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:29.905246 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-0108bbbe-b5c5-472a-b187-bac08b560940 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:29.907264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0108bbbe-b5c5-472a-b187-bac08b560940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:30.008533 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 117/225] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:30.171069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401d359-7252-4e38-ade9-e74310983880 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:30.171323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401d359-7252-4e38-ade9-e74310983880 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:30.171476 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401d359-7252-4e38-ade9-e74310983880 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:30.172928 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 109/226] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:30.179981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f838e57-3634-4f14-8bc3-f086d4420617 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:30.180356 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f838e57-3634-4f14-8bc3-f086d4420617 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:30.180854 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f838e57-3634-4f14-8bc3-f086d4420617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:30.210182 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-7f838e57-3634-4f14-8bc3-f086d4420617 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:30.212248 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7f838e57-3634-4f14-8bc3-f086d4420617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:30.274215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401d359-7252-4e38-ade9-e74310983880 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:30.292943 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:30.303913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 118/227] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:30.370809 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:30.462093 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401d359-7252-4e38-ade9-e74310983880 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:30.468872 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabfa7d-ceee-410d-bbe8-a504721bfabd None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:30.469196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabfa7d-ceee-410d-bbe8-a504721bfabd None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:30.469429 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabfa7d-ceee-410d-bbe8-a504721bfabd None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:30.470829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 119/228] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:35:30.476366 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a39855-08ef-41c2-b060-b6b826e66057 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:30.476728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a39855-08ef-41c2-b060-b6b826e66057 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:30.476958 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a39855-08ef-41c2-b060-b6b826e66057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:30.496046 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-81a39855-08ef-41c2-b060-b6b826e66057 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:30.497257 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-81a39855-08ef-41c2-b060-b6b826e66057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:30.522190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b98340-8650-4df2-acdb-f37447d9e5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:30.523389 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98340-8650-4df2-acdb-f37447d9e5ca None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:30.523680 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98340-8650-4df2-acdb-f37447d9e5ca None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:30.523980 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98340-8650-4df2-acdb-f37447d9e5ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:30.535683 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b98340-8650-4df2-acdb-f37447d9e5ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-c9b98340-8650-4df2-acdb-f37447d9e5ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:30.540644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:30.541555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:30.542328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:30.542328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:30.542328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:30.542328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:30.542328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:30.542328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:30.542328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:30.542328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:30.542328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:30.542328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:30.542328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 13:35:30.542328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:30.542328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 110/229] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:35:30.547764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b98340-8650-4df2-acdb-f37447d9e5ca None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:30.576532 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 120/230] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:30.577184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a39855-08ef-41c2-b060-b6b826e66057 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:30.662284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b98340-8650-4df2-acdb-f37447d9e5ca None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:30.677255 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:30.686767 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a39855-08ef-41c2-b060-b6b826e66057 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:30.700745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9846d2-772e-4bdd-acaf-22d33a29f973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:30.701365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9846d2-772e-4bdd-acaf-22d33a29f973 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:30.701646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9846d2-772e-4bdd-acaf-22d33a29f973 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:30.701910 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9846d2-772e-4bdd-acaf-22d33a29f973 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:30.702567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9846d2-772e-4bdd-acaf-22d33a29f973 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:30.702868 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9846d2-772e-4bdd-acaf-22d33a29f973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:30.704307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9846d2-772e-4bdd-acaf-22d33a29f973 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:30.716160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 111/231] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:30.718268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:30.719342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9846d2-772e-4bdd-acaf-22d33a29f973 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:30.728679 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfdc96-8365-4599-8556-9defe23a0005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:30.729573 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfdc96-8365-4599-8556-9defe23a0005 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:30.730063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfdc96-8365-4599-8556-9defe23a0005 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:30.730504 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfdc96-8365-4599-8556-9defe23a0005 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:30.731792 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfdc96-8365-4599-8556-9defe23a0005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:30.733761 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfdc96-8365-4599-8556-9defe23a0005 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:30.736481 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 121/232] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:30.742248 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfdc96-8365-4599-8556-9defe23a0005 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:30.834343 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9846d2-772e-4bdd-acaf-22d33a29f973 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:30.869746 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b06b5-2fc1-4288-8d1f-26813a3a1b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:30.870442 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b06b5-2fc1-4288-8d1f-26813a3a1b37 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:30.870690 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b06b5-2fc1-4288-8d1f-26813a3a1b37 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:30.870941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b06b5-2fc1-4288-8d1f-26813a3a1b37 None admin] PATH_INFO: `/v3/users/f287c0d6f68648ca834d4998bd41a35a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:30.883362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfdc96-8365-4599-8556-9defe23a0005 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:30.888294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b06b5-2fc1-4288-8d1f-26813a3a1b37 None admin] RBAC: Authorizing `identity:get_user(user_id=f287c0d6f68648ca834d4998bd41a35a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:30.889844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b06b5-2fc1-4288-8d1f-26813a3a1b37 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:30.891617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 112/233] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:30 2022] GET /identity/v3/users/f287c0d6f68648ca834d4998bd41a35a => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:30.897476 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b06b5-2fc1-4288-8d1f-26813a3a1b37 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:30.924728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a00204-d566-4afa-be20-daf47257d54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:30.925382 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a00204-d566-4afa-be20-daf47257d54a None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:30.925655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a00204-d566-4afa-be20-daf47257d54a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:30.925914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a00204-d566-4afa-be20-daf47257d54a None admin] PATH_INFO: `/v3/groups` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:30.926577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a00204-d566-4afa-be20-daf47257d54a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:30.928126 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a00204-d566-4afa-be20-daf47257d54a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:30.932696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:30.947006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 122/234] 10.222.0.36 () {60 vars in 1134 bytes} [Sat Oct 15 13:35:30 2022] POST /identity/v3/groups => generated 229 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:35:30.947006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a00204-d566-4afa-be20-daf47257d54a None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:31.037962 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:31.056608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b06b5-2fc1-4288-8d1f-26813a3a1b37 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:31.105593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0061916-e627-4eaf-8f45-0284f00b18de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:31.106089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a00204-d566-4afa-be20-daf47257d54a None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:31.106267 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0061916-e627-4eaf-8f45-0284f00b18de None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:31.106521 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0061916-e627-4eaf-8f45-0284f00b18de None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:31.106799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0061916-e627-4eaf-8f45-0284f00b18de None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:31.120669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0061916-e627-4eaf-8f45-0284f00b18de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f0061916-e627-4eaf-8f45-0284f00b18de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:31.125783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:31.127157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:31.128546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:31.128546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:31.128546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:31.128546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:31.128546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:31.128546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:31.128546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:31.128546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:31.128546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:31.128546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:31.128546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:31.128546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:31.128546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 113/235] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:31.133402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0061916-e627-4eaf-8f45-0284f00b18de None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:31.155180 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadefb1a-19bd-48ce-8247-558fa78a1cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:31.156194 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadefb1a-19bd-48ce-8247-558fa78a1cf7 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:31.156470 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadefb1a-19bd-48ce-8247-558fa78a1cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:31.156742 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadefb1a-19bd-48ce-8247-558fa78a1cf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:31.157517 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadefb1a-19bd-48ce-8247-558fa78a1cf7 None admin] RBAC: Adding query filter params (role.id=ba24faa1d7cb45f58bcb3bade9357c4c, user.id=f287c0d6f68648ca834d4998bd41a35a, scope.system=all) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:31.158047 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadefb1a-19bd-48ce-8247-558fa78a1cf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:31.159567 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadefb1a-19bd-48ce-8247-558fa78a1cf7 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:31.167841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 123/236] 10.222.0.36 () {56 vars in 1308 bytes} [Sat Oct 15 13:35:30 2022] GET /identity/v3/role_assignments?user.id=f287c0d6f68648ca834d4998bd41a35a&scope.system=all&role.id=ba24faa1d7cb45f58bcb3bade9357c4c => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:31.264626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:31.276743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0061916-e627-4eaf-8f45-0284f00b18de None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:31.318486 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5dfd2-01aa-40af-840f-691096f2488d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:31.319086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5dfd2-01aa-40af-840f-691096f2488d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:31.319358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5dfd2-01aa-40af-840f-691096f2488d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:31.319628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5dfd2-01aa-40af-840f-691096f2488d None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:31.320400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd5dfd2-01aa-40af-840f-691096f2488d None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:31.320722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd5dfd2-01aa-40af-840f-691096f2488d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:31.322175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd5dfd2-01aa-40af-840f-691096f2488d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:31.332767 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 114/237] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:31.339161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadefb1a-19bd-48ce-8247-558fa78a1cf7 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:31.473423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadefb1a-19bd-48ce-8247-558fa78a1cf7 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:31.497265 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:31.521046 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457198c-c5f4-4377-97fb-0fad76c84727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:31.521708 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457198c-c5f4-4377-97fb-0fad76c84727 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:31.521985 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457198c-c5f4-4377-97fb-0fad76c84727 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:31.522254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457198c-c5f4-4377-97fb-0fad76c84727 None admin] PATH_INFO: `/v3/users/8ff1b1beacd84db3948ade79a5042be0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:31.523575 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457198c-c5f4-4377-97fb-0fad76c84727 None admin] RBAC: Authorizing `identity:get_user(user_id=8ff1b1beacd84db3948ade79a5042be0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:31.526555 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457198c-c5f4-4377-97fb-0fad76c84727 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:31.528388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 124/238] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:31 2022] GET /identity/v3/users/8ff1b1beacd84db3948ade79a5042be0 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:31.535920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5dfd2-01aa-40af-840f-691096f2488d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:31.647336 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5dfd2-01aa-40af-840f-691096f2488d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:31.684375 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1429cf-887d-437e-b720-b6509890b264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:31.685044 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1429cf-887d-437e-b720-b6509890b264 None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:31.685319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1429cf-887d-437e-b720-b6509890b264 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:31.685589 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1429cf-887d-437e-b720-b6509890b264 None admin] PATH_INFO: `/v3/system/users/8ff1b1beacd84db3948ade79a5042be0/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:31.687006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1429cf-887d-437e-b720-b6509890b264 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8ff1b1beacd84db3948ade79a5042be0, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:31.688429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1429cf-887d-437e-b720-b6509890b264 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:31.701706 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 115/239] 10.222.0.36 () {58 vars in 1334 bytes} [Sat Oct 15 13:35:31 2022] PUT /identity/v3/system/users/8ff1b1beacd84db3948ade79a5042be0/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:35:33.370995 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e37647-b39a-43bd-a4b5-5178a49162e4 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:33.371544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e37647-b39a-43bd-a4b5-5178a49162e4 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:33.372119 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e37647-b39a-43bd-a4b5-5178a49162e4 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:33.374546 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 125/240] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:33.381093 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c85e6-7106-4825-9f8a-9ece9beb5f65 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:33.381679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c85e6-7106-4825-9f8a-9ece9beb5f65 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:33.381679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c85e6-7106-4825-9f8a-9ece9beb5f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:33.402696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b77c85e6-7106-4825-9f8a-9ece9beb5f65 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:33.404024 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b77c85e6-7106-4825-9f8a-9ece9beb5f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:33.404978 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2f369-fa90-483c-b73b-e1259cd6642e None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:33.405465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2f369-fa90-483c-b73b-e1259cd6642e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:33.405957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2f369-fa90-483c-b73b-e1259cd6642e None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:33.408810 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 126/241] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:35:33.414086 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf2124-4e78-4c19-baeb-9a854d342cf0 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:33.414526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf2124-4e78-4c19-baeb-9a854d342cf0 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:33.414674 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf2124-4e78-4c19-baeb-9a854d342cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:33.430955 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:33.436502 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-77cf2124-4e78-4c19-baeb-9a854d342cf0 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:33.437758 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-77cf2124-4e78-4c19-baeb-9a854d342cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:33.509085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 116/242] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:33.554227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 127/243] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:33.878735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c85e6-7106-4825-9f8a-9ece9beb5f65 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:33.910618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf2124-4e78-4c19-baeb-9a854d342cf0 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:34.056106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c85e6-7106-4825-9f8a-9ece9beb5f65 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:34.058495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf2124-4e78-4c19-baeb-9a854d342cf0 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:34.077079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:34.099203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a5dd9-fae8-44ba-8fb4-3ac7f0a7f67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:34.099848 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a5dd9-fae8-44ba-8fb4-3ac7f0a7f67e None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:34.100163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a5dd9-fae8-44ba-8fb4-3ac7f0a7f67e None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:34.100430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a5dd9-fae8-44ba-8fb4-3ac7f0a7f67e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:34.106514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fa6ed-23d3-4bc0-b2a9-e5a74be08fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:34.107242 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fa6ed-23d3-4bc0-b2a9-e5a74be08fc1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:34.107568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fa6ed-23d3-4bc0-b2a9-e5a74be08fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:34.107917 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fa6ed-23d3-4bc0-b2a9-e5a74be08fc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:34.109151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fa6ed-23d3-4bc0-b2a9-e5a74be08fc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:34.111851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fa6ed-23d3-4bc0-b2a9-e5a74be08fc1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:34.114318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 117/244] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:34.114860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c8f28-7e8c-496c-9bf9-8094dc31e3ce None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:34.115129 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c8f28-7e8c-496c-9bf9-8094dc31e3ce None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:34.115401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c8f28-7e8c-496c-9bf9-8094dc31e3ce None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:34.116880 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 118/245] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:34.119291 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a5dd9-fae8-44ba-8fb4-3ac7f0a7f67e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:34.121105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c8f28-7e8c-496c-9bf9-8094dc31e3ce None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-149a5dd9-fae8-44ba-8fb4-3ac7f0a7f67e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:34.129290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:34.131026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:34.133027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:34.133027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:34.133027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:34.133027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:34.133027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:34.133027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:34.133027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:34.133027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:34.133027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:34.133027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:34.133027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 13:35:34.133027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:34.133027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 128/246] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:35:34.135148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9427e-9d33-4b47-b4f5-07f3de1e4186 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:34.135148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9427e-9d33-4b47-b4f5-07f3de1e4186 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:34.135664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9427e-9d33-4b47-b4f5-07f3de1e4186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:34.147760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:34.164783 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-2ba9427e-9d33-4b47-b4f5-07f3de1e4186 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:34.166730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2ba9427e-9d33-4b47-b4f5-07f3de1e4186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:34.242870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 129/247] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:34.246610 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9427e-9d33-4b47-b4f5-07f3de1e4186 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:34.288668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c8f28-7e8c-496c-9bf9-8094dc31e3ce None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:34.344555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdf968-c5b2-45a1-81a7-e6e6e3ec24f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:34.345497 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdf968-c5b2-45a1-81a7-e6e6e3ec24f5 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:34.345937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdf968-c5b2-45a1-81a7-e6e6e3ec24f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:34.346616 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdf968-c5b2-45a1-81a7-e6e6e3ec24f5 None admin] PATH_INFO: `/v3/groups` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:34.347667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdf968-c5b2-45a1-81a7-e6e6e3ec24f5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:34.350163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdf968-c5b2-45a1-81a7-e6e6e3ec24f5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:34.377211 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 119/248] 10.222.0.36 () {60 vars in 1134 bytes} [Sat Oct 15 13:35:34 2022] POST /identity/v3/groups => generated 226 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:35:34.409570 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9427e-9d33-4b47-b4f5-07f3de1e4186 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:34.422751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:34.473431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a853bbc-6298-4ab9-b580-0af11fc806a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:34.474426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a853bbc-6298-4ab9-b580-0af11fc806a0 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:34.474866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a853bbc-6298-4ab9-b580-0af11fc806a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:34.475304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a853bbc-6298-4ab9-b580-0af11fc806a0 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:34.476465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a853bbc-6298-4ab9-b580-0af11fc806a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:34.476948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a853bbc-6298-4ab9-b580-0af11fc806a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:34.479384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a853bbc-6298-4ab9-b580-0af11fc806a0 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:34.494796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 130/249] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:34.503178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdf968-c5b2-45a1-81a7-e6e6e3ec24f5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:34.552585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a853bbc-6298-4ab9-b580-0af11fc806a0 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:34.635667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:34.675251 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdf968-c5b2-45a1-81a7-e6e6e3ec24f5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:34.715630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a853bbc-6298-4ab9-b580-0af11fc806a0 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:34.736614 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3726d-0cd1-4575-8546-a865beca0ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:34.737645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3726d-0cd1-4575-8546-a865beca0ca2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:34.738097 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3726d-0cd1-4575-8546-a865beca0ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:34.738486 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a3726d-0cd1-4575-8546-a865beca0ca2 None admin] PATH_INFO: `/v3/users/f287c0d6f68648ca834d4998bd41a35a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:34.740110 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3726d-0cd1-4575-8546-a865beca0ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=f287c0d6f68648ca834d4998bd41a35a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:34.741749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a3726d-0cd1-4575-8546-a865beca0ca2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:34.744799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 120/250] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:34 2022] GET /identity/v3/users/f287c0d6f68648ca834d4998bd41a35a => generated 293 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:34.752436 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3726d-0cd1-4575-8546-a865beca0ca2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:34.784560 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f829dd-b3dc-4b37-9f00-acf6aea9e190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:34.786315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f829dd-b3dc-4b37-9f00-acf6aea9e190 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:34.786315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f829dd-b3dc-4b37-9f00-acf6aea9e190 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:34.786831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f829dd-b3dc-4b37-9f00-acf6aea9e190 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:34.808362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f829dd-b3dc-4b37-9f00-acf6aea9e190 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-56f829dd-b3dc-4b37-9f00-acf6aea9e190 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:34.815553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:34.816526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:34.817279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:34.817279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:34.817279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:34.817279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:34.817279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:34.817279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:34.817279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:34.817279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:34.817279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:34.817279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:34.817279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:34.817279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:34.819288 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 131/251] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:35:34.829702 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f829dd-b3dc-4b37-9f00-acf6aea9e190 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:34.899638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:34.950481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a3726d-0cd1-4575-8546-a865beca0ca2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:35.018047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f499844-a686-4c9d-9d40-bdd42a3fd59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:35.019620 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f499844-a686-4c9d-9d40-bdd42a3fd59a None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:35.020340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f499844-a686-4c9d-9d40-bdd42a3fd59a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:35.021011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f499844-a686-4c9d-9d40-bdd42a3fd59a None admin] PATH_INFO: `/v3/system/users/f287c0d6f68648ca834d4998bd41a35a/roles/ba24faa1d7cb45f58bcb3bade9357c4c` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:35.025199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f499844-a686-4c9d-9d40-bdd42a3fd59a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f287c0d6f68648ca834d4998bd41a35a, role_id=ba24faa1d7cb45f58bcb3bade9357c4c)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:35.025428 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f829dd-b3dc-4b37-9f00-acf6aea9e190 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:35.028323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f499844-a686-4c9d-9d40-bdd42a3fd59a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:35.048405 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 121/252] 10.222.0.36 () {58 vars in 1334 bytes} [Sat Oct 15 13:35:34 2022] PUT /identity/v3/system/users/f287c0d6f68648ca834d4998bd41a35a/roles/ba24faa1d7cb45f58bcb3bade9357c4c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:35:35.066616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663da27-598f-475e-96da-0bf72436dc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:35.067256 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663da27-598f-475e-96da-0bf72436dc49 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:35.067558 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663da27-598f-475e-96da-0bf72436dc49 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:35.067795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663da27-598f-475e-96da-0bf72436dc49 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:35.068498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663da27-598f-475e-96da-0bf72436dc49 None admin] RBAC: Adding query filter params (name=member) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:35.068796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663da27-598f-475e-96da-0bf72436dc49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:35.070221 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663da27-598f-475e-96da-0bf72436dc49 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:35.080329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 132/253] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:35.086995 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f499844-a686-4c9d-9d40-bdd42a3fd59a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:35.124414 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:35.311519 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f499844-a686-4c9d-9d40-bdd42a3fd59a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:35.382393 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885efd3-f3ba-451e-80a2-a843fb637579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:35.383452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885efd3-f3ba-451e-80a2-a843fb637579 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:35.383950 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885efd3-f3ba-451e-80a2-a843fb637579 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:35.384420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885efd3-f3ba-451e-80a2-a843fb637579 None admin] PATH_INFO: `/v3/users/8ff1b1beacd84db3948ade79a5042be0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:35.386438 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885efd3-f3ba-451e-80a2-a843fb637579 None admin] RBAC: Authorizing `identity:get_user(user_id=8ff1b1beacd84db3948ade79a5042be0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:35.389356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885efd3-f3ba-451e-80a2-a843fb637579 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:35.394710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 122/254] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:35 2022] GET /identity/v3/users/8ff1b1beacd84db3948ade79a5042be0 => generated 293 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:35.399409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663da27-598f-475e-96da-0bf72436dc49 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:35.573960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:35.589980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663da27-598f-475e-96da-0bf72436dc49 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:35.652488 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d48f25-ff9d-4a94-93a7-97aa38dd20fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:35.653492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d48f25-ff9d-4a94-93a7-97aa38dd20fd None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:35.653894 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d48f25-ff9d-4a94-93a7-97aa38dd20fd None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:35.654290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d48f25-ff9d-4a94-93a7-97aa38dd20fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:35.655426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d48f25-ff9d-4a94-93a7-97aa38dd20fd None admin] RBAC: Adding query filter params (role.id=ad5c2cf379e34fdcb85aa27f334e3fc0, user.id=8ff1b1beacd84db3948ade79a5042be0, scope.system=all) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:35.655899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d48f25-ff9d-4a94-93a7-97aa38dd20fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:35.658911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d48f25-ff9d-4a94-93a7-97aa38dd20fd None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:35.682213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 133/255] 10.222.0.36 () {56 vars in 1308 bytes} [Sat Oct 15 13:35:35 2022] GET /identity/v3/role_assignments?user.id=8ff1b1beacd84db3948ade79a5042be0&scope.system=all&role.id=ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 510 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:37.303264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e264c-dbc7-4521-a4e1-7a7e6a7ba359 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:37.304179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e264c-dbc7-4521-a4e1-7a7e6a7ba359 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:37.304179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e264c-dbc7-4521-a4e1-7a7e6a7ba359 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:37.306556 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 123/256] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:37.313792 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c980e2-33e4-4d06-87b9-7cac24d7daad None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:37.315387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c980e2-33e4-4d06-87b9-7cac24d7daad None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:37.315387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c980e2-33e4-4d06-87b9-7cac24d7daad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:37.354412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-44c980e2-33e4-4d06-87b9-7cac24d7daad None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:37.356592 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-44c980e2-33e4-4d06-87b9-7cac24d7daad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:37.492565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 134/257] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 13:35:38.055133 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e264c-dbc7-4521-a4e1-7a7e6a7ba359 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:38.082426 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e264c-dbc7-4521-a4e1-7a7e6a7ba359 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:38.094655 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820099a5-c0dd-45ba-a326-23a5f6a75105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:38.095753 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820099a5-c0dd-45ba-a326-23a5f6a75105 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:38.096262 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820099a5-c0dd-45ba-a326-23a5f6a75105 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:38.096720 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820099a5-c0dd-45ba-a326-23a5f6a75105 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:38.122759 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820099a5-c0dd-45ba-a326-23a5f6a75105 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-820099a5-c0dd-45ba-a326-23a5f6a75105 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:38.134318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:38.137001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:38.140060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:38.140060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:38.140060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:38.140060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:38.140060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:38.140060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:38.140060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:38.140060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:38.140060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:38.140060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:38.140060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:38.140060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:38.140060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 124/258] 10.222.0.36 () {56 vars in 1120 bytes} [Sat Oct 15 13:35:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 82 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:38.147908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c980e2-33e4-4d06-87b9-7cac24d7daad None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:38.167465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c980e2-33e4-4d06-87b9-7cac24d7daad None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:38.178952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0891f60-8343-4044-938d-d39aa7ca43d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:38.180004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0891f60-8343-4044-938d-d39aa7ca43d5 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:38.180454 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0891f60-8343-4044-938d-d39aa7ca43d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:38.180984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0891f60-8343-4044-938d-d39aa7ca43d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:38.182151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0891f60-8343-4044-938d-d39aa7ca43d5 admin admin] RBAC: Adding query filter params (name=member) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:38.182776 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0891f60-8343-4044-938d-d39aa7ca43d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:38.186072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: WARNING py.warnings [None req-d0891f60-8343-4044-938d-d39aa7ca43d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:35:38.186072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: warnings.warn(msg) Oct 15 13:35:38.186072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: Oct 15 13:35:38.187749 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0891f60-8343-4044-938d-d39aa7ca43d5 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:38.210476 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 135/259] 10.222.0.36 () {56 vars in 1122 bytes} [Sat Oct 15 13:35:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:38.218831 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820099a5-c0dd-45ba-a326-23a5f6a75105 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:38.241927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820099a5-c0dd-45ba-a326-23a5f6a75105 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:38.250581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee189e-1eec-40cf-aacb-f6fea7f96c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:38.251683 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee189e-1eec-40cf-aacb-f6fea7f96c71 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:38.252187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee189e-1eec-40cf-aacb-f6fea7f96c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:38.252641 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee189e-1eec-40cf-aacb-f6fea7f96c71 admin admin] PATH_INFO: `/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:38.269087 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee189e-1eec-40cf-aacb-f6fea7f96c71 admin admin] RBAC: Authorizing `identity:get_group(group_id=2d138fcb16e74352ae73777c8cdd3b1e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:38.272681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-0dee189e-1eec-40cf-aacb-f6fea7f96c71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:35:38.272681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Oct 15 13:35:38.272681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: Oct 15 13:35:38.275253 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee189e-1eec-40cf-aacb-f6fea7f96c71 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:38.278598 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 125/260] 10.222.0.36 () {56 vars in 1201 bytes} [Sat Oct 15 13:35:38 2022] GET /identity/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e => generated 226 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:38.288530 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0891f60-8343-4044-938d-d39aa7ca43d5 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:38.309227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0891f60-8343-4044-938d-d39aa7ca43d5 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:38.320808 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a5b11-9265-41cc-859f-97f0fe42e247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:38.322020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a5b11-9265-41cc-859f-97f0fe42e247 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:38.322444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a5b11-9265-41cc-859f-97f0fe42e247 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:38.323015 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a5b11-9265-41cc-859f-97f0fe42e247 admin admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:38.325275 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a5b11-9265-41cc-859f-97f0fe42e247 admin admin] RBAC: Authorizing `identity:get_project(project_id=b3780a3b8fe742949173e4396724f2e0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:38.329877 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a5b11-9265-41cc-859f-97f0fe42e247 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:38.333498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 136/261] 10.222.0.36 () {56 vars in 1207 bytes} [Sat Oct 15 13:35:38 2022] GET /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0 => generated 298 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:38.343501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee189e-1eec-40cf-aacb-f6fea7f96c71 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:38.362622 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee189e-1eec-40cf-aacb-f6fea7f96c71 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:38.374870 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbbf36-6ab9-4b71-940f-fc6e7fa2d0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:38.376169 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbbf36-6ab9-4b71-940f-fc6e7fa2d0fe admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:38.377307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbbf36-6ab9-4b71-940f-fc6e7fa2d0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:38.377994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbbf36-6ab9-4b71-940f-fc6e7fa2d0fe admin admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0/groups/2d138fcb16e74352ae73777c8cdd3b1e/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:38.383981 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbbf36-6ab9-4b71-940f-fc6e7fa2d0fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3780a3b8fe742949173e4396724f2e0, group_id=2d138fcb16e74352ae73777c8cdd3b1e, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:38.383981 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-88bbbf36-6ab9-4b71-940f-fc6e7fa2d0fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:35:38.383981 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Oct 15 13:35:38.383981 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: Oct 15 13:35:38.388189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbbf36-6ab9-4b71-940f-fc6e7fa2d0fe admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:38.426655 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 126/262] 10.222.0.36 () {58 vars in 1463 bytes} [Sat Oct 15 13:35:38 2022] PUT /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0/groups/2d138fcb16e74352ae73777c8cdd3b1e/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:35:38.743559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd882b-92a6-4438-ae5f-4e2e7169a647 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:38.743734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd882b-92a6-4438-ae5f-4e2e7169a647 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:38.744054 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd882b-92a6-4438-ae5f-4e2e7169a647 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:38.745863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 137/263] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:38.757875 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf2832-fbfc-4a24-b1bd-051e9950ec8b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:38.758212 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf2832-fbfc-4a24-b1bd-051e9950ec8b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:38.758422 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf2832-fbfc-4a24-b1bd-051e9950ec8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:38.786346 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-83bf2832-fbfc-4a24-b1bd-051e9950ec8b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:38.787570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-83bf2832-fbfc-4a24-b1bd-051e9950ec8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:38.842248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:38.869239 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 127/264] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 13:35:39.197181 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd882b-92a6-4438-ae5f-4e2e7169a647 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:39.214695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd882b-92a6-4438-ae5f-4e2e7169a647 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:39.219219 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8828265-a14d-4193-8afb-9e51f5b30cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:39.219905 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8828265-a14d-4193-8afb-9e51f5b30cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:39.220275 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8828265-a14d-4193-8afb-9e51f5b30cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:39.220509 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8828265-a14d-4193-8afb-9e51f5b30cc7 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:39.239898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8828265-a14d-4193-8afb-9e51f5b30cc7 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-c8828265-a14d-4193-8afb-9e51f5b30cc7 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:39.246290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:39.249553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:39.252682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:39.252682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:39.252682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:39.252682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:39.252682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:39.252682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:39.252682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:39.252682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:39.252682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:39.252682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:39.252682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:39.252682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:39.252682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 138/265] 10.222.0.36 () {56 vars in 1135 bytes} [Sat Oct 15 13:35:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:39.255104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf2832-fbfc-4a24-b1bd-051e9950ec8b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:39.263347 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf2832-fbfc-4a24-b1bd-051e9950ec8b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:39.271019 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3443e-5498-461d-9cbd-90e6f55d89f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:39.271663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3443e-5498-461d-9cbd-90e6f55d89f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:39.271982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3443e-5498-461d-9cbd-90e6f55d89f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:39.272252 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3443e-5498-461d-9cbd-90e6f55d89f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:39.272938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3443e-5498-461d-9cbd-90e6f55d89f5 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:39.273233 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3443e-5498-461d-9cbd-90e6f55d89f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:39.274276 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-95b3443e-5498-461d-9cbd-90e6f55d89f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:35:39.274276 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Oct 15 13:35:39.274276 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: Oct 15 13:35:39.275184 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3443e-5498-461d-9cbd-90e6f55d89f5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:39.288938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 128/266] 10.222.0.36 () {56 vars in 1132 bytes} [Sat Oct 15 13:35:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:39.294208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8828265-a14d-4193-8afb-9e51f5b30cc7 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:39.304889 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8828265-a14d-4193-8afb-9e51f5b30cc7 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:39.312096 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946e7e1-df3f-412d-a8ec-98ecc3acdb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:39.312755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946e7e1-df3f-412d-a8ec-98ecc3acdb2a admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:39.313004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946e7e1-df3f-412d-a8ec-98ecc3acdb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:39.313282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946e7e1-df3f-412d-a8ec-98ecc3acdb2a admin admin] PATH_INFO: `/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:39.314515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946e7e1-df3f-412d-a8ec-98ecc3acdb2a admin admin] RBAC: Authorizing `identity:get_group(group_id=2d138fcb16e74352ae73777c8cdd3b1e)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:39.315549 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: WARNING py.warnings [None req-7946e7e1-df3f-412d-a8ec-98ecc3acdb2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:35:39.315549 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: warnings.warn(msg) Oct 15 13:35:39.315549 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: Oct 15 13:35:39.316878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946e7e1-df3f-412d-a8ec-98ecc3acdb2a admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:39.318572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 139/267] 10.222.0.36 () {56 vars in 1201 bytes} [Sat Oct 15 13:35:39 2022] GET /identity/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e => generated 226 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:39.324359 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3443e-5498-461d-9cbd-90e6f55d89f5 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:39.341788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3443e-5498-461d-9cbd-90e6f55d89f5 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:39.350518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a834b-44b0-4104-bf42-10dbf921f084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:39.351484 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a834b-44b0-4104-bf42-10dbf921f084 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:39.351484 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a834b-44b0-4104-bf42-10dbf921f084 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:39.351944 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a834b-44b0-4104-bf42-10dbf921f084 admin admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:39.352867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a834b-44b0-4104-bf42-10dbf921f084 admin admin] RBAC: Authorizing `identity:get_project(project_id=b3780a3b8fe742949173e4396724f2e0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:39.354964 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a834b-44b0-4104-bf42-10dbf921f084 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:39.356698 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 129/268] 10.222.0.36 () {56 vars in 1207 bytes} [Sat Oct 15 13:35:39 2022] GET /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0 => generated 298 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:39.363428 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946e7e1-df3f-412d-a8ec-98ecc3acdb2a admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:39.373621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946e7e1-df3f-412d-a8ec-98ecc3acdb2a admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:39.384347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8767d5-9aea-480a-a255-7b85abaeb942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:39.385036 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8767d5-9aea-480a-a255-7b85abaeb942 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:39.385309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8767d5-9aea-480a-a255-7b85abaeb942 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:39.385602 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8767d5-9aea-480a-a255-7b85abaeb942 admin admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0/groups/2d138fcb16e74352ae73777c8cdd3b1e/roles/8a864244c21042b88accaa76b9117665` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:39.387769 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8767d5-9aea-480a-a255-7b85abaeb942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3780a3b8fe742949173e4396724f2e0, group_id=2d138fcb16e74352ae73777c8cdd3b1e, role_id=8a864244c21042b88accaa76b9117665)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:39.388844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: WARNING py.warnings [None req-5c8767d5-9aea-480a-a255-7b85abaeb942 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:35:39.388844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: warnings.warn(msg) Oct 15 13:35:39.388844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: Oct 15 13:35:39.390802 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8767d5-9aea-480a-a255-7b85abaeb942 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:39.414561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 140/269] 10.222.0.36 () {58 vars in 1463 bytes} [Sat Oct 15 13:35:39 2022] PUT /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0/groups/2d138fcb16e74352ae73777c8cdd3b1e/roles/8a864244c21042b88accaa76b9117665 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:35:39.552893 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27804a1d-ba31-4596-a6b2-e4aaf1dbcb20 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:39.553282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27804a1d-ba31-4596-a6b2-e4aaf1dbcb20 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:39.553501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27804a1d-ba31-4596-a6b2-e4aaf1dbcb20 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:39.555072 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 130/270] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:39.561296 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb31c3-f042-4de4-a9af-e6838fcc77f3 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:39.561678 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb31c3-f042-4de4-a9af-e6838fcc77f3 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:39.561853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb31c3-f042-4de4-a9af-e6838fcc77f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:39.582264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-9abb31c3-f042-4de4-a9af-e6838fcc77f3 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:39.583472 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9abb31c3-f042-4de4-a9af-e6838fcc77f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:39.629833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990c56b-088c-41e1-b135-ea5779d9e345 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:39.630390 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990c56b-088c-41e1-b135-ea5779d9e345 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:39.630665 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990c56b-088c-41e1-b135-ea5779d9e345 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:39.632473 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 131/271] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:39.638500 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3f344-f0d3-4156-9773-2197cd35ce85 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:39.638826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3f344-f0d3-4156-9773-2197cd35ce85 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:39.639029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3f344-f0d3-4156-9773-2197cd35ce85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:39.677863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-fdf3f344-f0d3-4156-9773-2197cd35ce85 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:39.679135 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fdf3f344-f0d3-4156-9773-2197cd35ce85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:39.680315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:39.715388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 141/272] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:39.799540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 132/273] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 15 13:35:39.835564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8b778-134f-4068-82ed-10ab37c69aca None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:39.835944 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8b778-134f-4068-82ed-10ab37c69aca None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:39.836096 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8b778-134f-4068-82ed-10ab37c69aca None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:39.837628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 142/274] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:39.846007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff8b9e-b34c-45c9-bde4-f874b4606697 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:39.846007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff8b9e-b34c-45c9-bde4-f874b4606697 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:39.846007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff8b9e-b34c-45c9-bde4-f874b4606697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:39.868902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-fbff8b9e-b34c-45c9-bde4-f874b4606697 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:39.870455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fbff8b9e-b34c-45c9-bde4-f874b4606697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:39.882568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 133/275] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 13:35:40.088563 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8b778-134f-4068-82ed-10ab37c69aca None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:40.225003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8b778-134f-4068-82ed-10ab37c69aca None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:40.266976 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aafa35-7be3-4f0c-8ad2-874a68579626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:40.267670 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aafa35-7be3-4f0c-8ad2-874a68579626 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:40.268005 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aafa35-7be3-4f0c-8ad2-874a68579626 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:40.268432 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aafa35-7be3-4f0c-8ad2-874a68579626 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:40.286178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aafa35-7be3-4f0c-8ad2-874a68579626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-19aafa35-7be3-4f0c-8ad2-874a68579626 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:40.291124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:40.292342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:40.293161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:40.293161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:40.293161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:40.293161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:40.293161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:40.293161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:40.293161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:40.293161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:40.293161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:40.293161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:40.293161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 15 13:35:40.293161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:40.293161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 143/276] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:40.298488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff8b9e-b34c-45c9-bde4-f874b4606697 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:40.437300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff8b9e-b34c-45c9-bde4-f874b4606697 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:40.462816 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:40.482575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a7b19-0fce-438a-9fdd-f477f7fe766d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:40.483214 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a7b19-0fce-438a-9fdd-f477f7fe766d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:40.483485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a7b19-0fce-438a-9fdd-f477f7fe766d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:40.483747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a7b19-0fce-438a-9fdd-f477f7fe766d None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:40.484800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a7b19-0fce-438a-9fdd-f477f7fe766d None admin] RBAC: Adding query filter params (name=reader) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:40.485160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a7b19-0fce-438a-9fdd-f477f7fe766d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:40.486824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a7b19-0fce-438a-9fdd-f477f7fe766d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:40.500370 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 134/277] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:40.506362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aafa35-7be3-4f0c-8ad2-874a68579626 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:40.531350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a7b19-0fce-438a-9fdd-f477f7fe766d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:40.555439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a7b19-0fce-438a-9fdd-f477f7fe766d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:40.566092 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdadc08-c304-4799-93c6-bfd1a166e931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:40.566753 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdadc08-c304-4799-93c6-bfd1a166e931 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:40.567033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdadc08-c304-4799-93c6-bfd1a166e931 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:40.567299 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdadc08-c304-4799-93c6-bfd1a166e931 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:40.593157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:40.594330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdadc08-c304-4799-93c6-bfd1a166e931 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-8fdadc08-c304-4799-93c6-bfd1a166e931 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:40.601188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:40.602154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:40.602935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:40.602935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:40.602935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:40.602935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:40.602935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:40.602935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:40.602935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:40.602935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:40.602935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:40.602935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:40.602935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:40.602935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:40.604086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 135/278] 10.222.0.36 () {56 vars in 1135 bytes} [Sat Oct 15 13:35:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 73 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:40.614030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdadc08-c304-4799-93c6-bfd1a166e931 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:40.632408 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdadc08-c304-4799-93c6-bfd1a166e931 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:40.646068 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b9fec-602d-46a9-9fea-336fc2656963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:40.647079 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b9fec-602d-46a9-9fea-336fc2656963 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:40.647350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b9fec-602d-46a9-9fea-336fc2656963 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:40.648750 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b9fec-602d-46a9-9fea-336fc2656963 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:40.672899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b9fec-602d-46a9-9fea-336fc2656963 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1f1b9fec-602d-46a9-9fea-336fc2656963 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:40.678538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:40.679481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:40.684758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:40.684758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:40.684758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:40.684758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:40.684758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:40.684758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:40.684758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:40.684758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:40.684758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:40.684758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:40.684758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:40.684758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:40.687222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 136/279] 10.222.0.36 () {56 vars in 1120 bytes} [Sat Oct 15 13:35:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 77 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:40.688979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b9fec-602d-46a9-9fea-336fc2656963 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:40.713099 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aafa35-7be3-4f0c-8ad2-874a68579626 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:40.714794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b9fec-602d-46a9-9fea-336fc2656963 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:40.724183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0809e-89ce-492f-945d-7c5d9b9a7736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:40.725196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0809e-89ce-492f-945d-7c5d9b9a7736 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:40.725682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0809e-89ce-492f-945d-7c5d9b9a7736 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:40.726156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0809e-89ce-492f-945d-7c5d9b9a7736 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:40.727023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0809e-89ce-492f-945d-7c5d9b9a7736 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:40.727536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0809e-89ce-492f-945d-7c5d9b9a7736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:40.735537 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0809e-89ce-492f-945d-7c5d9b9a7736 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:40.754292 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 137/280] 10.222.0.36 () {56 vars in 1132 bytes} [Sat Oct 15 13:35:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:40.757667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0809e-89ce-492f-945d-7c5d9b9a7736 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:40.773482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ce8fb-7218-4df2-8139-33226dcf8c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:40.774534 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ce8fb-7218-4df2-8139-33226dcf8c54 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:40.775011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ce8fb-7218-4df2-8139-33226dcf8c54 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:40.775483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ce8fb-7218-4df2-8139-33226dcf8c54 None admin] PATH_INFO: `/v3/users/f287c0d6f68648ca834d4998bd41a35a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:40.776930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ce8fb-7218-4df2-8139-33226dcf8c54 None admin] RBAC: Authorizing `identity:get_user(user_id=f287c0d6f68648ca834d4998bd41a35a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:40.778372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0809e-89ce-492f-945d-7c5d9b9a7736 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:40.779043 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ce8fb-7218-4df2-8139-33226dcf8c54 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:40.782063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 144/281] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:35:40 2022] GET /identity/v3/users/f287c0d6f68648ca834d4998bd41a35a => generated 293 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:40.788551 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ce8fb-7218-4df2-8139-33226dcf8c54 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:40.800377 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e235c-451f-459a-9728-986567f8c438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:40.802621 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e235c-451f-459a-9728-986567f8c438 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:40.803363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e235c-451f-459a-9728-986567f8c438 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:40.805162 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e235c-451f-459a-9728-986567f8c438 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:40.806080 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e235c-451f-459a-9728-986567f8c438 admin admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:40.807049 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e235c-451f-459a-9728-986567f8c438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:40.808223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ce8fb-7218-4df2-8139-33226dcf8c54 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:40.810163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e235c-451f-459a-9728-986567f8c438 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:40.820936 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a470b-3677-4b90-b5da-f30fa897eaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:40.821960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a470b-3677-4b90-b5da-f30fa897eaf1 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:40.822434 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a470b-3677-4b90-b5da-f30fa897eaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:40.822895 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a470b-3677-4b90-b5da-f30fa897eaf1 admin admin] PATH_INFO: `/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:40.825448 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a470b-3677-4b90-b5da-f30fa897eaf1 admin admin] RBAC: Authorizing `identity:get_group(group_id=2d138fcb16e74352ae73777c8cdd3b1e)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:40.827620 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a470b-3677-4b90-b5da-f30fa897eaf1 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:40.829820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 145/282] 10.222.0.36 () {56 vars in 1201 bytes} [Sat Oct 15 13:35:40 2022] GET /identity/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e => generated 226 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:40.831994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 138/283] 10.222.0.36 () {56 vars in 1122 bytes} [Sat Oct 15 13:35:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:40.833096 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e235c-451f-459a-9728-986567f8c438 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:40.839959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a470b-3677-4b90-b5da-f30fa897eaf1 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:40.859160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a470b-3677-4b90-b5da-f30fa897eaf1 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:40.872896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f5fce-818c-4ca0-89bf-b9ad0ca6cb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:40.873568 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f5fce-818c-4ca0-89bf-b9ad0ca6cb70 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:40.873847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f5fce-818c-4ca0-89bf-b9ad0ca6cb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:40.876423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f5fce-818c-4ca0-89bf-b9ad0ca6cb70 admin admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:40.879376 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f5fce-818c-4ca0-89bf-b9ad0ca6cb70 admin admin] RBAC: Authorizing `identity:get_project(project_id=34af9b28c3d641ddba8bde542506dc85)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:40.886657 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f5fce-818c-4ca0-89bf-b9ad0ca6cb70 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:40.888676 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 146/284] 10.222.0.36 () {56 vars in 1207 bytes} [Sat Oct 15 13:35:40 2022] GET /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85 => generated 302 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:40.896347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f5fce-818c-4ca0-89bf-b9ad0ca6cb70 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:40.922908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f5fce-818c-4ca0-89bf-b9ad0ca6cb70 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:40.935747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da0845-50fb-4eeb-9f3d-4829bae5ee6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:40.936536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da0845-50fb-4eeb-9f3d-4829bae5ee6a admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:40.936817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da0845-50fb-4eeb-9f3d-4829bae5ee6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:40.937088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da0845-50fb-4eeb-9f3d-4829bae5ee6a admin admin] PATH_INFO: `/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:40.938605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da0845-50fb-4eeb-9f3d-4829bae5ee6a admin admin] RBAC: Authorizing `identity:get_group(group_id=2d138fcb16e74352ae73777c8cdd3b1e)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:40.943800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da0845-50fb-4eeb-9f3d-4829bae5ee6a admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:40.950868 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 147/285] 10.222.0.36 () {56 vars in 1201 bytes} [Sat Oct 15 13:35:40 2022] GET /identity/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e => generated 226 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:40.951754 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da0845-50fb-4eeb-9f3d-4829bae5ee6a admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:40.969406 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da0845-50fb-4eeb-9f3d-4829bae5ee6a admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:40.985593 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c07f4-10c5-4779-9956-6122cb40e23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:40.986340 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c07f4-10c5-4779-9956-6122cb40e23a admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:40.986614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c07f4-10c5-4779-9956-6122cb40e23a admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:40.986886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c07f4-10c5-4779-9956-6122cb40e23a admin admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85/groups/2d138fcb16e74352ae73777c8cdd3b1e/roles/8a864244c21042b88accaa76b9117665` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:40.993888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c07f4-10c5-4779-9956-6122cb40e23a admin admin] RBAC: Authorizing `identity:create_grant(project_id=34af9b28c3d641ddba8bde542506dc85, group_id=2d138fcb16e74352ae73777c8cdd3b1e, role_id=8a864244c21042b88accaa76b9117665)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:40.996834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c07f4-10c5-4779-9956-6122cb40e23a admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:41.022651 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 148/286] 10.222.0.36 () {58 vars in 1463 bytes} [Sat Oct 15 13:35:40 2022] PUT /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85/groups/2d138fcb16e74352ae73777c8cdd3b1e/roles/8a864244c21042b88accaa76b9117665 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:35:41.023349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c07f4-10c5-4779-9956-6122cb40e23a admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:41.054949 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e235c-451f-459a-9728-986567f8c438 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:41.117731 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92353a71-1347-473e-ae50-7413ba777d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:41.118389 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92353a71-1347-473e-ae50-7413ba777d63 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:41.118666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92353a71-1347-473e-ae50-7413ba777d63 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:41.118935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92353a71-1347-473e-ae50-7413ba777d63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:41.119692 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92353a71-1347-473e-ae50-7413ba777d63 None admin] RBAC: Adding query filter params (role.id=ba24faa1d7cb45f58bcb3bade9357c4c, user.id=f287c0d6f68648ca834d4998bd41a35a, scope.system=all) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:41.120056 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92353a71-1347-473e-ae50-7413ba777d63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:41.121548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92353a71-1347-473e-ae50-7413ba777d63 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:41.123371 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:41.131334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c07f4-10c5-4779-9956-6122cb40e23a admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:41.138024 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 139/287] 10.222.0.36 () {56 vars in 1308 bytes} [Sat Oct 15 13:35:40 2022] GET /identity/v3/role_assignments?user.id=f287c0d6f68648ca834d4998bd41a35a&scope.system=all&role.id=ba24faa1d7cb45f58bcb3bade9357c4c => generated 510 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:41.139498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41c7f1-a16d-4d9b-a033-c38d1d998d9a None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:41.139785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41c7f1-a16d-4d9b-a033-c38d1d998d9a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:41.142754 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41c7f1-a16d-4d9b-a033-c38d1d998d9a None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:41.144374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 140/288] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:41.147838 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527998d-1c8d-4018-b46f-b6eba32ec9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:41.148558 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527998d-1c8d-4018-b46f-b6eba32ec9dd admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:41.149028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527998d-1c8d-4018-b46f-b6eba32ec9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:41.149195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527998d-1c8d-4018-b46f-b6eba32ec9dd admin admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:41.151394 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc1532-38c7-4a3a-a9e5-f5528e73377e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:41.152245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc1532-38c7-4a3a-a9e5-f5528e73377e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:41.152727 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc1532-38c7-4a3a-a9e5-f5528e73377e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:41.156857 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527998d-1c8d-4018-b46f-b6eba32ec9dd admin admin] RBAC: Authorizing `identity:get_project(project_id=34af9b28c3d641ddba8bde542506dc85)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:41.159553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527998d-1c8d-4018-b46f-b6eba32ec9dd admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:41.161750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 149/289] 10.222.0.36 () {56 vars in 1207 bytes} [Sat Oct 15 13:35:41 2022] GET /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:41.167401 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527998d-1c8d-4018-b46f-b6eba32ec9dd admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:41.182389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527998d-1c8d-4018-b46f-b6eba32ec9dd admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:41.184459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-36dc1532-38c7-4a3a-a9e5-f5528e73377e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:41.187274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-36dc1532-38c7-4a3a-a9e5-f5528e73377e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:41.201828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd5db7-b02a-411b-a322-58648869fc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:41.202420 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd5db7-b02a-411b-a322-58648869fc83 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:41.202726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd5db7-b02a-411b-a322-58648869fc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:41.202967 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd5db7-b02a-411b-a322-58648869fc83 admin admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85/groups/2d138fcb16e74352ae73777c8cdd3b1e/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:41.205823 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 141/290] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 15 13:35:41.212896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd5db7-b02a-411b-a322-58648869fc83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34af9b28c3d641ddba8bde542506dc85, group_id=2d138fcb16e74352ae73777c8cdd3b1e, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:41.215649 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd5db7-b02a-411b-a322-58648869fc83 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:41.242858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 150/291] 10.222.0.36 () {58 vars in 1463 bytes} [Sat Oct 15 13:35:41 2022] PUT /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85/groups/2d138fcb16e74352ae73777c8cdd3b1e/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:35:41.666221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc1532-38c7-4a3a-a9e5-f5528e73377e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:41.668467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0bf99-5719-43b7-95b4-99f357ddf330 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:41.669255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0bf99-5719-43b7-95b4-99f357ddf330 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:41.669759 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0bf99-5719-43b7-95b4-99f357ddf330 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:41.671331 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 151/292] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:41.677991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c198a87-409c-45d1-9fc8-e37b59a13d48 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:41.678615 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c198a87-409c-45d1-9fc8-e37b59a13d48 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:41.679078 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c198a87-409c-45d1-9fc8-e37b59a13d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:41.679265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc1532-38c7-4a3a-a9e5-f5528e73377e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:41.707114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-8c198a87-409c-45d1-9fc8-e37b59a13d48 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:41.709595 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8c198a87-409c-45d1-9fc8-e37b59a13d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:41.748653 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:41.801936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c1576-74a2-4193-9956-dc4cef6342f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:41.802922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c1576-74a2-4193-9956-dc4cef6342f6 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:41.803360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c1576-74a2-4193-9956-dc4cef6342f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:41.803782 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c1576-74a2-4193-9956-dc4cef6342f6 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:41.834614 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c1576-74a2-4193-9956-dc4cef6342f6 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f16c1576-74a2-4193-9956-dc4cef6342f6 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:41.840134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:41.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:41.841878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:41.841878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:41.841878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:41.841878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:41.841878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:41.841878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:41.841878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:41.841878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:41.841878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:41.841878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:41.841878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:35:41.841878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:41.843935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 142/293] 10.222.0.36 () {56 vars in 1117 bytes} [Sat Oct 15 13:35:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:41.852959 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c1576-74a2-4193-9956-dc4cef6342f6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:41.868104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c1576-74a2-4193-9956-dc4cef6342f6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:41.880776 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 152/294] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:41.893456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2df8e-712a-4695-af6e-d0e35581d551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:41.894099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2df8e-712a-4695-af6e-d0e35581d551 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:41.894373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2df8e-712a-4695-af6e-d0e35581d551 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:41.894639 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2df8e-712a-4695-af6e-d0e35581d551 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:41.895325 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2df8e-712a-4695-af6e-d0e35581d551 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:41.895622 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2df8e-712a-4695-af6e-d0e35581d551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:41.897411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2df8e-712a-4695-af6e-d0e35581d551 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:41.913507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 143/295] 10.222.0.36 () {56 vars in 1120 bytes} [Sat Oct 15 13:35:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:41.919384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c198a87-409c-45d1-9fc8-e37b59a13d48 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:41.931125 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c198a87-409c-45d1-9fc8-e37b59a13d48 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:41.938614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b0759-5a55-4007-b6e3-a1a4a486b5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:41.939293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b0759-5a55-4007-b6e3-a1a4a486b5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:41.939586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b0759-5a55-4007-b6e3-a1a4a486b5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:41.939971 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b0759-5a55-4007-b6e3-a1a4a486b5c3 admin admin] PATH_INFO: `/v3/groups/cd841ff9a88a49e9a441a1da338f905e` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:41.949445 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b0759-5a55-4007-b6e3-a1a4a486b5c3 admin admin] RBAC: Authorizing `identity:get_group(group_id=cd841ff9a88a49e9a441a1da338f905e)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:41.951418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b0759-5a55-4007-b6e3-a1a4a486b5c3 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:41.953867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 153/296] 10.222.0.36 () {56 vars in 1201 bytes} [Sat Oct 15 13:35:41 2022] GET /identity/v3/groups/cd841ff9a88a49e9a441a1da338f905e => generated 229 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:41.959748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2df8e-712a-4695-af6e-d0e35581d551 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:41.971188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2df8e-712a-4695-af6e-d0e35581d551 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:41.979468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbd563-c9e2-4faa-a465-35e9c5b42dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:41.980168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbd563-c9e2-4faa-a465-35e9c5b42dbf admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:41.980439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbd563-c9e2-4faa-a465-35e9c5b42dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:41.980703 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbd563-c9e2-4faa-a465-35e9c5b42dbf admin admin] PATH_INFO: `/v3/projects/443104cd704b44c6867e1bc065e90d39` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:41.981905 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbd563-c9e2-4faa-a465-35e9c5b42dbf admin admin] RBAC: Authorizing `identity:get_project(project_id=443104cd704b44c6867e1bc065e90d39)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:41.983722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbd563-c9e2-4faa-a465-35e9c5b42dbf admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:41.985828 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 144/297] 10.222.0.36 () {56 vars in 1207 bytes} [Sat Oct 15 13:35:41 2022] GET /identity/v3/projects/443104cd704b44c6867e1bc065e90d39 => generated 344 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:41.992213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b0759-5a55-4007-b6e3-a1a4a486b5c3 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:42.003061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b0759-5a55-4007-b6e3-a1a4a486b5c3 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:42.012183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc046f3-7d1c-4dd2-8d63-a05aabc449d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:42.012879 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc046f3-7d1c-4dd2-8d63-a05aabc449d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:42.013153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc046f3-7d1c-4dd2-8d63-a05aabc449d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:42.013417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc046f3-7d1c-4dd2-8d63-a05aabc449d3 admin admin] PATH_INFO: `/v3/projects/443104cd704b44c6867e1bc065e90d39/groups/cd841ff9a88a49e9a441a1da338f905e/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:42.021528 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc046f3-7d1c-4dd2-8d63-a05aabc449d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=443104cd704b44c6867e1bc065e90d39, group_id=cd841ff9a88a49e9a441a1da338f905e, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:42.024228 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc046f3-7d1c-4dd2-8d63-a05aabc449d3 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:42.046614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 154/298] 10.222.0.36 () {58 vars in 1463 bytes} [Sat Oct 15 13:35:41 2022] PUT /identity/v3/projects/443104cd704b44c6867e1bc065e90d39/groups/cd841ff9a88a49e9a441a1da338f905e/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:35:42.431032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbd563-c9e2-4faa-a465-35e9c5b42dbf admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:42.601043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbd563-c9e2-4faa-a465-35e9c5b42dbf admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:42.632047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:42.672788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772058e5-cdce-4737-a984-3e21dddee425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:42.674053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772058e5-cdce-4737-a984-3e21dddee425 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:42.674549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772058e5-cdce-4737-a984-3e21dddee425 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:42.675035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772058e5-cdce-4737-a984-3e21dddee425 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:42.699755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772058e5-cdce-4737-a984-3e21dddee425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-772058e5-cdce-4737-a984-3e21dddee425 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:42.709613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:42.710611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:42.711385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:42.711385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:42.711385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:42.711385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:42.711385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:42.711385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:42.711385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:42.711385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:42.711385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:42.711385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:42.711385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:42.711385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:42.713920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 145/299] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:35:42.722274 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc046f3-7d1c-4dd2-8d63-a05aabc449d3 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:42.938582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc046f3-7d1c-4dd2-8d63-a05aabc449d3 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:42.958099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132a15c-f586-4a40-b34b-3fb106c41d7c None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:42.958427 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132a15c-f586-4a40-b34b-3fb106c41d7c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:42.958896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132a15c-f586-4a40-b34b-3fb106c41d7c None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:42.961430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 146/300] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:35:42.980363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c57643-e058-4c3f-be3c-a89bc275f7a7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:42.980831 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c57643-e058-4c3f-be3c-a89bc275f7a7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:42.981290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c57643-e058-4c3f-be3c-a89bc275f7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:43.027020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:43.036472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f4c57643-e058-4c3f-be3c-a89bc275f7a7 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:43.038436 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f4c57643-e058-4c3f-be3c-a89bc275f7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:43.046982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0010e5b-270b-4c30-aa79-5be18bb8d105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:43.047866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0010e5b-270b-4c30-aa79-5be18bb8d105 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:43.048477 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0010e5b-270b-4c30-aa79-5be18bb8d105 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:43.048933 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0010e5b-270b-4c30-aa79-5be18bb8d105 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:43.049791 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0010e5b-270b-4c30-aa79-5be18bb8d105 None admin] RBAC: Adding query filter params (name=member) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:43.050752 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0010e5b-270b-4c30-aa79-5be18bb8d105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:43.053102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0010e5b-270b-4c30-aa79-5be18bb8d105 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:43.084729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 155/301] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:43.112628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0010e5b-270b-4c30-aa79-5be18bb8d105 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:43.256271 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 147/302] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:43.323855 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0010e5b-270b-4c30-aa79-5be18bb8d105 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:43.399294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6b886-f89b-4c47-900b-77baec6abd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:43.400378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6b886-f89b-4c47-900b-77baec6abd7b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:43.400863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6b886-f89b-4c47-900b-77baec6abd7b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:43.401337 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6b886-f89b-4c47-900b-77baec6abd7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:43.524481 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:43.539150 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6b886-f89b-4c47-900b-77baec6abd7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:43.541993 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6b886-f89b-4c47-900b-77baec6abd7b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:43.739473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 156/303] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:35:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:35:43.749152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c57643-e058-4c3f-be3c-a89bc275f7a7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:43.821478 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6b886-f89b-4c47-900b-77baec6abd7b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:43.978999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c57643-e058-4c3f-be3c-a89bc275f7a7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:43.982168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:44.066558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cb18b-2e5a-4cd5-b698-9f3065760424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:44.067904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb18b-2e5a-4cd5-b698-9f3065760424 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:44.068376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb18b-2e5a-4cd5-b698-9f3065760424 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:44.068829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb18b-2e5a-4cd5-b698-9f3065760424 None admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:44.071101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68cb18b-2e5a-4cd5-b698-9f3065760424 None admin] RBAC: Authorizing `identity:get_project(project_id=b3780a3b8fe742949173e4396724f2e0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:44.074391 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68cb18b-2e5a-4cd5-b698-9f3065760424 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:44.078113 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 148/304] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:43 2022] GET /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0 => generated 298 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:44.085653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cb18b-2e5a-4cd5-b698-9f3065760424 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:44.094071 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:44.107735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6b886-f89b-4c47-900b-77baec6abd7b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:44.185365 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c01bc0-ab79-41a8-b958-30d747222bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:44.186028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c01bc0-ab79-41a8-b958-30d747222bff None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:44.186304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c01bc0-ab79-41a8-b958-30d747222bff None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:44.186574 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c01bc0-ab79-41a8-b958-30d747222bff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:44.199844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c01bc0-ab79-41a8-b958-30d747222bff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-b9c01bc0-ab79-41a8-b958-30d747222bff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:44.205495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:44.207223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:44.208766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:44.208766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:44.208766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:44.208766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:44.208766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:44.208766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:44.208766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:44.208766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:44.208766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:44.208766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:44.208766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:44.208766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:44.208766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 157/305] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:35:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:35:44.208766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998171c1-10ab-4105-bfdb-a5205f48120d None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:44.210161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998171c1-10ab-4105-bfdb-a5205f48120d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:44.210161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998171c1-10ab-4105-bfdb-a5205f48120d None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:44.211681 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 158/306] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:44.219748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907295b6-d7c3-4c2f-ae3d-4e22fbd878db None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:44.220117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907295b6-d7c3-4c2f-ae3d-4e22fbd878db None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:44.220465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907295b6-d7c3-4c2f-ae3d-4e22fbd878db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:44.242882 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-907295b6-d7c3-4c2f-ae3d-4e22fbd878db None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:44.244157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-907295b6-d7c3-4c2f-ae3d-4e22fbd878db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:44.323255 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cb18b-2e5a-4cd5-b698-9f3065760424 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:44.372216 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 159/307] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:44.373798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907295b6-d7c3-4c2f-ae3d-4e22fbd878db None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:44.376815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55666b23-3873-44b5-b723-344e7f1e4d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:44.377775 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55666b23-3873-44b5-b723-344e7f1e4d89 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:44.378442 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55666b23-3873-44b5-b723-344e7f1e4d89 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:44.378734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55666b23-3873-44b5-b723-344e7f1e4d89 None admin] PATH_INFO: `/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:44.381124 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55666b23-3873-44b5-b723-344e7f1e4d89 None admin] RBAC: Authorizing `identity:get_group(group_id=2d138fcb16e74352ae73777c8cdd3b1e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:44.383931 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55666b23-3873-44b5-b723-344e7f1e4d89 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:44.395290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 149/308] 10.222.0.36 () {56 vars in 1180 bytes} [Sat Oct 15 13:35:44 2022] GET /identity/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e => generated 226 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:44.402261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55666b23-3873-44b5-b723-344e7f1e4d89 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:44.495339 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:44.537548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:44.572613 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907295b6-d7c3-4c2f-ae3d-4e22fbd878db None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:44.641063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55666b23-3873-44b5-b723-344e7f1e4d89 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:44.646238 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce602e95-aace-45f4-bf3c-8ca9e93db8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:44.647755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce602e95-aace-45f4-bf3c-8ca9e93db8db None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:44.648550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce602e95-aace-45f4-bf3c-8ca9e93db8db None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:44.649309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce602e95-aace-45f4-bf3c-8ca9e93db8db None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:44.650872 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce602e95-aace-45f4-bf3c-8ca9e93db8db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:44.651697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce602e95-aace-45f4-bf3c-8ca9e93db8db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:44.655393 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce602e95-aace-45f4-bf3c-8ca9e93db8db None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:44.677279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 160/309] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:35:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:44.687221 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce602e95-aace-45f4-bf3c-8ca9e93db8db None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:44.711931 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58168403-c369-4743-9322-8269c3822494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:44.711931 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58168403-c369-4743-9322-8269c3822494 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:44.711931 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58168403-c369-4743-9322-8269c3822494 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:44.711931 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58168403-c369-4743-9322-8269c3822494 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:44.712576 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58168403-c369-4743-9322-8269c3822494 None admin] RBAC: Adding query filter params (group.id=2d138fcb16e74352ae73777c8cdd3b1e, role.id=ad5c2cf379e34fdcb85aa27f334e3fc0, scope.project.id=b3780a3b8fe742949173e4396724f2e0) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:44.716567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58168403-c369-4743-9322-8269c3822494 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:44.719341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58168403-c369-4743-9322-8269c3822494 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:44.733725 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 150/310] 10.222.0.36 () {56 vars in 1376 bytes} [Sat Oct 15 13:35:44 2022] GET /identity/v3/role_assignments?group.id=2d138fcb16e74352ae73777c8cdd3b1e&scope.project.id=b3780a3b8fe742949173e4396724f2e0&role.id=ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 611 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:44.840430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58168403-c369-4743-9322-8269c3822494 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:44.989871 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce602e95-aace-45f4-bf3c-8ca9e93db8db None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:45.073195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58168403-c369-4743-9322-8269c3822494 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:45.075364 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:45.076138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6184f9-6aaf-4756-b418-e355a6cc0073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:45.079623 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6184f9-6aaf-4756-b418-e355a6cc0073 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:45.080427 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6184f9-6aaf-4756-b418-e355a6cc0073 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:45.081163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6184f9-6aaf-4756-b418-e355a6cc0073 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:45.128101 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:45.130296 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e2aba-f185-40fd-9b8c-d55928be5e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:45.131268 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e2aba-f185-40fd-9b8c-d55928be5e6f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:45.131440 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e2aba-f185-40fd-9b8c-d55928be5e6f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:45.131737 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e2aba-f185-40fd-9b8c-d55928be5e6f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:45.153047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e2aba-f185-40fd-9b8c-d55928be5e6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-780e2aba-f185-40fd-9b8c-d55928be5e6f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:45.159434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:45.161279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:45.162859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:45.162859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:45.162859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:45.162859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:45.162859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:45.162859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:45.162859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:45.162859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:45.162859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:45.162859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:45.162859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:45.162859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:45.162859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 151/311] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:35:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:35:45.172967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e2aba-f185-40fd-9b8c-d55928be5e6f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:45.254043 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6184f9-6aaf-4756-b418-e355a6cc0073 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:45.257174 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6184f9-6aaf-4756-b418-e355a6cc0073 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:45.365756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e2aba-f185-40fd-9b8c-d55928be5e6f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:45.425359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 161/312] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:35:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 15 13:35:45.426829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ae7b1-c008-467d-bdcf-c9cc5ed86215 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:45.427181 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ae7b1-c008-467d-bdcf-c9cc5ed86215 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:45.427482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ae7b1-c008-467d-bdcf-c9cc5ed86215 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:45.429025 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 162/313] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:45.435355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f1f32-9b6f-4a9a-866e-5cc5b6c71745 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:45.435655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f1f32-9b6f-4a9a-866e-5cc5b6c71745 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:45.435994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f1f32-9b6f-4a9a-866e-5cc5b6c71745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:45.439624 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618d5eb-2284-4686-94be-cb7eebde29b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:45.440861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618d5eb-2284-4686-94be-cb7eebde29b4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:45.441341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618d5eb-2284-4686-94be-cb7eebde29b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:45.442037 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618d5eb-2284-4686-94be-cb7eebde29b4 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:45.443485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618d5eb-2284-4686-94be-cb7eebde29b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:45.444194 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618d5eb-2284-4686-94be-cb7eebde29b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:45.447090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618d5eb-2284-4686-94be-cb7eebde29b4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:45.457663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-3c6f1f32-9b6f-4a9a-866e-5cc5b6c71745 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:45.458996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3c6f1f32-9b6f-4a9a-866e-5cc5b6c71745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:45.465509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 152/314] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:35:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:45.467180 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618d5eb-2284-4686-94be-cb7eebde29b4 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:45.515033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:45.525757 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:45.562717 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 163/315] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:45.563960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f1f32-9b6f-4a9a-866e-5cc5b6c71745 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:45.654264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618d5eb-2284-4686-94be-cb7eebde29b4 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:45.682527 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f1f32-9b6f-4a9a-866e-5cc5b6c71745 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:45.697693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8dd68-0a57-463b-add0-87328a31509f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:45.698575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8dd68-0a57-463b-add0-87328a31509f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:45.698949 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8dd68-0a57-463b-add0-87328a31509f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:45.699312 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8dd68-0a57-463b-add0-87328a31509f None admin] PATH_INFO: `/v3/projects/b3780a3b8fe742949173e4396724f2e0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:45.700896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8dd68-0a57-463b-add0-87328a31509f None admin] RBAC: Authorizing `identity:get_project(project_id=b3780a3b8fe742949173e4396724f2e0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:45.702653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8dd68-0a57-463b-add0-87328a31509f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:45.705157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 153/316] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:45 2022] GET /identity/v3/projects/b3780a3b8fe742949173e4396724f2e0 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:45.713743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8dd68-0a57-463b-add0-87328a31509f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:45.725849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c1cf5-51c0-491c-b7e5-8bf30a29eab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:45.726773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c1cf5-51c0-491c-b7e5-8bf30a29eab5 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:45.727283 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c1cf5-51c0-491c-b7e5-8bf30a29eab5 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:45.727764 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c1cf5-51c0-491c-b7e5-8bf30a29eab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:45.779729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:45.807050 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c1cf5-51c0-491c-b7e5-8bf30a29eab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:45.808942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c1cf5-51c0-491c-b7e5-8bf30a29eab5 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:45.832781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:45.842609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8dd68-0a57-463b-add0-87328a31509f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:45.903334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12eff3f-e168-4f69-a105-f61f650b401e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:45.904476 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12eff3f-e168-4f69-a105-f61f650b401e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:45.904948 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12eff3f-e168-4f69-a105-f61f650b401e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:45.905412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12eff3f-e168-4f69-a105-f61f650b401e None admin] PATH_INFO: `/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:45.909622 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12eff3f-e168-4f69-a105-f61f650b401e None admin] RBAC: Authorizing `identity:get_group(group_id=2d138fcb16e74352ae73777c8cdd3b1e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:45.912548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12eff3f-e168-4f69-a105-f61f650b401e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:45.915924 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 154/317] 10.222.0.36 () {56 vars in 1180 bytes} [Sat Oct 15 13:35:45 2022] GET /identity/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:45.917852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42437a-20ad-428b-8619-02ed188887ac None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:45.918327 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42437a-20ad-428b-8619-02ed188887ac None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:45.918805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42437a-20ad-428b-8619-02ed188887ac None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:45.929732 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 155/318] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:45 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:45.931234 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42437a-20ad-428b-8619-02ed188887ac None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:45.949453 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 164/319] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:35:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:35:45.951172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281dd4f-0273-458a-b22e-96467324a89d None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:45.951673 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281dd4f-0273-458a-b22e-96467324a89d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:45.952255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281dd4f-0273-458a-b22e-96467324a89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:45.993123 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-b281dd4f-0273-458a-b22e-96467324a89d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:45.995328 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b281dd4f-0273-458a-b22e-96467324a89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:46.104251 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 165/320] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:46.104251 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281dd4f-0273-458a-b22e-96467324a89d None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:46.107827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42437a-20ad-428b-8619-02ed188887ac None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:46.130341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:46.171549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651eeffe-e70e-46b7-b017-034fdfcb7864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:46.172452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651eeffe-e70e-46b7-b017-034fdfcb7864 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:46.177352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651eeffe-e70e-46b7-b017-034fdfcb7864 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:46.177352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651eeffe-e70e-46b7-b017-034fdfcb7864 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:46.177352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651eeffe-e70e-46b7-b017-034fdfcb7864 None admin] RBAC: Adding query filter params (group.id=2d138fcb16e74352ae73777c8cdd3b1e, role.id=8a864244c21042b88accaa76b9117665, scope.project.id=b3780a3b8fe742949173e4396724f2e0) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:46.177352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651eeffe-e70e-46b7-b017-034fdfcb7864 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:46.177352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651eeffe-e70e-46b7-b017-034fdfcb7864 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:46.184382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 156/321] 10.222.0.36 () {56 vars in 1376 bytes} [Sat Oct 15 13:35:45 2022] GET /identity/v3/role_assignments?group.id=2d138fcb16e74352ae73777c8cdd3b1e&scope.project.id=b3780a3b8fe742949173e4396724f2e0&role.id=8a864244c21042b88accaa76b9117665 => generated 611 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:46.185352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651eeffe-e70e-46b7-b017-034fdfcb7864 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:46.225093 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:46.322146 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281dd4f-0273-458a-b22e-96467324a89d None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:46.361461 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651eeffe-e70e-46b7-b017-034fdfcb7864 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:46.373839 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a74e57-6ec7-46e2-972d-8a89cd5911b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:46.374845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a74e57-6ec7-46e2-972d-8a89cd5911b0 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:46.375383 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a74e57-6ec7-46e2-972d-8a89cd5911b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:46.375826 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a74e57-6ec7-46e2-972d-8a89cd5911b0 None admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:46.377621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a74e57-6ec7-46e2-972d-8a89cd5911b0 None admin] RBAC: Authorizing `identity:get_project(project_id=34af9b28c3d641ddba8bde542506dc85)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:46.379387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a74e57-6ec7-46e2-972d-8a89cd5911b0 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:46.382311 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 166/322] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:46 2022] GET /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:46.389753 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a74e57-6ec7-46e2-972d-8a89cd5911b0 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:46.416774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456cec5-27a5-43eb-8c83-3543f3f82ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:46.417275 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456cec5-27a5-43eb-8c83-3543f3f82ebd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:46.417554 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456cec5-27a5-43eb-8c83-3543f3f82ebd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:46.417822 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456cec5-27a5-43eb-8c83-3543f3f82ebd None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:46.434320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456cec5-27a5-43eb-8c83-3543f3f82ebd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-2456cec5-27a5-43eb-8c83-3543f3f82ebd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:46.442595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:46.443562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:46.444373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:46.444373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:46.444373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:46.444373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:46.444373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:46.444373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:46.444373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:46.444373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:46.444373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:46.444373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:46.444373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 15 13:35:46.444373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:46.444373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 157/323] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:46.445020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456cec5-27a5-43eb-8c83-3543f3f82ebd None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:46.508712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:46.546882 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a74e57-6ec7-46e2-972d-8a89cd5911b0 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:46.576985 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:46.594037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479c5e6-79ee-4ddb-bba2-992e29e53ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:46.594700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479c5e6-79ee-4ddb-bba2-992e29e53ecf None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:46.595036 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479c5e6-79ee-4ddb-bba2-992e29e53ecf None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:46.595311 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479c5e6-79ee-4ddb-bba2-992e29e53ecf None admin] PATH_INFO: `/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:46.597693 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479c5e6-79ee-4ddb-bba2-992e29e53ecf None admin] RBAC: Authorizing `identity:get_group(group_id=2d138fcb16e74352ae73777c8cdd3b1e)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:46.599246 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479c5e6-79ee-4ddb-bba2-992e29e53ecf None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:46.601056 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456cec5-27a5-43eb-8c83-3543f3f82ebd None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:46.601346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 167/324] 10.222.0.36 () {56 vars in 1180 bytes} [Sat Oct 15 13:35:46 2022] GET /identity/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:46.604209 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479c5e6-79ee-4ddb-bba2-992e29e53ecf None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:46.648087 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1dc851-a8ce-4d09-ac77-8fc1183265f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:46.649240 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dc851-a8ce-4d09-ac77-8fc1183265f2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:46.649685 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dc851-a8ce-4d09-ac77-8fc1183265f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:46.650045 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dc851-a8ce-4d09-ac77-8fc1183265f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:46.666771 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1dc851-a8ce-4d09-ac77-8fc1183265f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-bb1dc851-a8ce-4d09-ac77-8fc1183265f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:46.672193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:46.673899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:46.675439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:46.675439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:46.675439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:46.675439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:46.675439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:46.675439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:46.675439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:46.675439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:46.675439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:46.675439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:46.675439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:35:46.675439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:46.675439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 158/325] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:35:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:46.675439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1dc851-a8ce-4d09-ac77-8fc1183265f2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:46.755078 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479c5e6-79ee-4ddb-bba2-992e29e53ecf None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:46.793268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55157bb6-3687-447c-9b21-620f1db63255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:46.793893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55157bb6-3687-447c-9b21-620f1db63255 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:46.794171 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55157bb6-3687-447c-9b21-620f1db63255 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:46.794430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55157bb6-3687-447c-9b21-620f1db63255 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:46.795111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55157bb6-3687-447c-9b21-620f1db63255 None admin] RBAC: Adding query filter params (name=member) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:46.795410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55157bb6-3687-447c-9b21-620f1db63255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:46.796865 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55157bb6-3687-447c-9b21-620f1db63255 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:46.808293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 168/326] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:35:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:46.809081 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55157bb6-3687-447c-9b21-620f1db63255 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:46.813464 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:46.829944 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1dc851-a8ce-4d09-ac77-8fc1183265f2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:46.892424 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec59c42-d04f-401a-9c27-73f7a3699d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:46.893186 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec59c42-d04f-401a-9c27-73f7a3699d11 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:46.893315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec59c42-d04f-401a-9c27-73f7a3699d11 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:46.893663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec59c42-d04f-401a-9c27-73f7a3699d11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:46.894289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec59c42-d04f-401a-9c27-73f7a3699d11 None admin] RBAC: Adding query filter params (group.id=2d138fcb16e74352ae73777c8cdd3b1e, role.id=8a864244c21042b88accaa76b9117665, scope.project.id=34af9b28c3d641ddba8bde542506dc85) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:46.895504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec59c42-d04f-401a-9c27-73f7a3699d11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:46.896106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec59c42-d04f-401a-9c27-73f7a3699d11 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:46.903836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 159/327] 10.222.0.36 () {56 vars in 1376 bytes} [Sat Oct 15 13:35:46 2022] GET /identity/v3/role_assignments?group.id=2d138fcb16e74352ae73777c8cdd3b1e&scope.project.id=34af9b28c3d641ddba8bde542506dc85&role.id=8a864244c21042b88accaa76b9117665 => generated 611 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:46.905236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec59c42-d04f-401a-9c27-73f7a3699d11 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:46.918069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:46.980876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55157bb6-3687-447c-9b21-620f1db63255 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:47.035852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec59c42-d04f-401a-9c27-73f7a3699d11 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:47.049043 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac63de-1c61-4771-91c2-2da0a59086d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:47.051568 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac63de-1c61-4771-91c2-2da0a59086d7 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:47.052472 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac63de-1c61-4771-91c2-2da0a59086d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:47.053252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac63de-1c61-4771-91c2-2da0a59086d7 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:47.055121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac63de-1c61-4771-91c2-2da0a59086d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:47.055819 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac63de-1c61-4771-91c2-2da0a59086d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:47.058793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac63de-1c61-4771-91c2-2da0a59086d7 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:47.071486 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 169/328] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:47.077947 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac63de-1c61-4771-91c2-2da0a59086d7 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:47.083402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927276d-9761-4257-921b-76eaa30ec4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:47.084286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927276d-9761-4257-921b-76eaa30ec4f6 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:47.084721 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927276d-9761-4257-921b-76eaa30ec4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:47.085146 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927276d-9761-4257-921b-76eaa30ec4f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:47.168806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:47.191586 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927276d-9761-4257-921b-76eaa30ec4f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:47.193440 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927276d-9761-4257-921b-76eaa30ec4f6 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:47.205035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:47.214532 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac63de-1c61-4771-91c2-2da0a59086d7 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:47.250806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736bbad-230a-473e-a571-311a60cc536b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:47.251561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736bbad-230a-473e-a571-311a60cc536b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:47.251845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736bbad-230a-473e-a571-311a60cc536b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:47.252156 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736bbad-230a-473e-a571-311a60cc536b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:47.316710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 160/329] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:35:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:35:47.325264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927276d-9761-4257-921b-76eaa30ec4f6 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:47.325515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736bbad-230a-473e-a571-311a60cc536b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:47.326954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736bbad-230a-473e-a571-311a60cc536b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:47.418276 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:47.450948 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927276d-9761-4257-921b-76eaa30ec4f6 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:47.458881 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 170/330] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:35:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:35:47.464739 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736bbad-230a-473e-a571-311a60cc536b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:47.468697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:47.500183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273884de-e54a-4c42-9d5a-1ba7cf8a3ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:47.500828 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273884de-e54a-4c42-9d5a-1ba7cf8a3ef9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:47.501101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273884de-e54a-4c42-9d5a-1ba7cf8a3ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:47.501428 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273884de-e54a-4c42-9d5a-1ba7cf8a3ef9 None admin] PATH_INFO: `/v3/projects/34af9b28c3d641ddba8bde542506dc85` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:47.502637 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273884de-e54a-4c42-9d5a-1ba7cf8a3ef9 None admin] RBAC: Authorizing `identity:get_project(project_id=34af9b28c3d641ddba8bde542506dc85)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:47.504206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273884de-e54a-4c42-9d5a-1ba7cf8a3ef9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:47.505982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 161/331] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:47 2022] GET /identity/v3/projects/34af9b28c3d641ddba8bde542506dc85 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:47.514670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273884de-e54a-4c42-9d5a-1ba7cf8a3ef9 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:47.593976 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736bbad-230a-473e-a571-311a60cc536b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:47.634881 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21383f1-d8f2-4ef8-bbb1-793405d47ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:47.635608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21383f1-d8f2-4ef8-bbb1-793405d47ba1 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:47.635991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21383f1-d8f2-4ef8-bbb1-793405d47ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:47.636298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273884de-e54a-4c42-9d5a-1ba7cf8a3ef9 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:47.636454 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21383f1-d8f2-4ef8-bbb1-793405d47ba1 None admin] PATH_INFO: `/v3/projects/443104cd704b44c6867e1bc065e90d39` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:47.637914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21383f1-d8f2-4ef8-bbb1-793405d47ba1 None admin] RBAC: Authorizing `identity:get_project(project_id=443104cd704b44c6867e1bc065e90d39)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:47.639608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21383f1-d8f2-4ef8-bbb1-793405d47ba1 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:47.641474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 171/332] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:35:47 2022] GET /identity/v3/projects/443104cd704b44c6867e1bc065e90d39 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:47.650421 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21383f1-d8f2-4ef8-bbb1-793405d47ba1 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:47.685628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6013f62-7b21-4c5d-ab6f-3e8c5317f118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:47.686350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6013f62-7b21-4c5d-ab6f-3e8c5317f118 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:47.686714 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6013f62-7b21-4c5d-ab6f-3e8c5317f118 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:47.687135 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6013f62-7b21-4c5d-ab6f-3e8c5317f118 None admin] PATH_INFO: `/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:47.688992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6013f62-7b21-4c5d-ab6f-3e8c5317f118 None admin] RBAC: Authorizing `identity:get_group(group_id=2d138fcb16e74352ae73777c8cdd3b1e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:47.691530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6013f62-7b21-4c5d-ab6f-3e8c5317f118 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:47.694685 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 162/333] 10.222.0.36 () {56 vars in 1180 bytes} [Sat Oct 15 13:35:47 2022] GET /identity/v3/groups/2d138fcb16e74352ae73777c8cdd3b1e => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:47.701476 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6013f62-7b21-4c5d-ab6f-3e8c5317f118 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:47.701975 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:47.768548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:47.794825 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21383f1-d8f2-4ef8-bbb1-793405d47ba1 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:47.837560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6013f62-7b21-4c5d-ab6f-3e8c5317f118 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:47.839742 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e70db-9467-47fd-915b-3db17b743aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:47.840564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e70db-9467-47fd-915b-3db17b743aca None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:47.840974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e70db-9467-47fd-915b-3db17b743aca None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:47.841356 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e70db-9467-47fd-915b-3db17b743aca None admin] PATH_INFO: `/v3/groups/cd841ff9a88a49e9a441a1da338f905e` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:47.842708 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e70db-9467-47fd-915b-3db17b743aca None admin] RBAC: Authorizing `identity:get_group(group_id=cd841ff9a88a49e9a441a1da338f905e)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:47.844309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e70db-9467-47fd-915b-3db17b743aca None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:47.846136 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 172/334] 10.222.0.36 () {56 vars in 1180 bytes} [Sat Oct 15 13:35:47 2022] GET /identity/v3/groups/cd841ff9a88a49e9a441a1da338f905e => generated 229 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:47.853439 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e70db-9467-47fd-915b-3db17b743aca None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:47.892805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf1244-ee14-47b0-91ba-055e164261b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:47.893459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf1244-ee14-47b0-91ba-055e164261b0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:47.893700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf1244-ee14-47b0-91ba-055e164261b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:47.893968 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf1244-ee14-47b0-91ba-055e164261b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:47.894709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf1244-ee14-47b0-91ba-055e164261b0 None admin] RBAC: Adding query filter params (group.id=2d138fcb16e74352ae73777c8cdd3b1e, role.id=ad5c2cf379e34fdcb85aa27f334e3fc0, scope.project.id=34af9b28c3d641ddba8bde542506dc85) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:47.895012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf1244-ee14-47b0-91ba-055e164261b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:47.896621 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf1244-ee14-47b0-91ba-055e164261b0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:47.903176 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 163/335] 10.222.0.36 () {56 vars in 1376 bytes} [Sat Oct 15 13:35:47 2022] GET /identity/v3/role_assignments?group.id=2d138fcb16e74352ae73777c8cdd3b1e&scope.project.id=34af9b28c3d641ddba8bde542506dc85&role.id=ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:47.984130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e70db-9467-47fd-915b-3db17b743aca None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:47.986436 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:48.031144 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2074ec-9323-4d57-aa1d-225b9e88c50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:48.031775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2074ec-9323-4d57-aa1d-225b9e88c50a None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:48.032077 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2074ec-9323-4d57-aa1d-225b9e88c50a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:48.032337 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2074ec-9323-4d57-aa1d-225b9e88c50a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:48.033223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2074ec-9323-4d57-aa1d-225b9e88c50a None admin] RBAC: Adding query filter params (group.id=cd841ff9a88a49e9a441a1da338f905e, role.id=57740dc2ebe1429294f20dd51592b82a, scope.project.id=443104cd704b44c6867e1bc065e90d39) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:48.033529 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2074ec-9323-4d57-aa1d-225b9e88c50a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:48.035011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2074ec-9323-4d57-aa1d-225b9e88c50a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:48.042068 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 173/336] 10.222.0.36 () {56 vars in 1376 bytes} [Sat Oct 15 13:35:47 2022] GET /identity/v3/role_assignments?group.id=cd841ff9a88a49e9a441a1da338f905e&scope.project.id=443104cd704b44c6867e1bc065e90d39&role.id=57740dc2ebe1429294f20dd51592b82a => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:50.951978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b297f5b-f70a-4831-b4d7-aca8fda23be9 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:50.951978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b297f5b-f70a-4831-b4d7-aca8fda23be9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:50.951978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b297f5b-f70a-4831-b4d7-aca8fda23be9 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:50.953466 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 164/337] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:50.964816 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da78002-4391-46a8-8bdb-81b4b7f2c974 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:50.966201 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da78002-4391-46a8-8bdb-81b4b7f2c974 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:50.966913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da78002-4391-46a8-8bdb-81b4b7f2c974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:51.007965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-3da78002-4391-46a8-8bdb-81b4b7f2c974 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:51.010430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3da78002-4391-46a8-8bdb-81b4b7f2c974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:51.157969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 174/338] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:51.437211 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0cf74-68db-429c-94a6-2605175a0c53 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:51.438078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0cf74-68db-429c-94a6-2605175a0c53 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:51.438831 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0cf74-68db-429c-94a6-2605175a0c53 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:51.442253 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 165/339] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:51.450072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cd33b-c109-49e3-af45-9e6ee8fe864e None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:51.450349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cd33b-c109-49e3-af45-9e6ee8fe864e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:51.450771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cd33b-c109-49e3-af45-9e6ee8fe864e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:51.463797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf8432-346c-472d-8c16-40f8f7b24818 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:51.464778 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf8432-346c-472d-8c16-40f8f7b24818 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:51.465672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf8432-346c-472d-8c16-40f8f7b24818 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:51.469023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 166/340] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:51.475020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cd3a6-0b41-4b00-be11-39c5cfa57ede None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:51.475530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cd3a6-0b41-4b00-be11-39c5cfa57ede None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:51.476000 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cd3a6-0b41-4b00-be11-39c5cfa57ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:51.481915 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-404cd33b-c109-49e3-af45-9e6ee8fe864e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:51.483423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-404cd33b-c109-49e3-af45-9e6ee8fe864e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:51.499976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f54cd3a6-0b41-4b00-be11-39c5cfa57ede None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:51.501601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f54cd3a6-0b41-4b00-be11-39c5cfa57ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:51.652615 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 167/341] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:51.659821 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54cd3a6-0b41-4b00-be11-39c5cfa57ede None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:51.665348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 175/342] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:51.724561 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:51.800208 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54cd3a6-0b41-4b00-be11-39c5cfa57ede None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:51.847723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a366f33-a017-49d1-a2a6-8ee26b84e6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:51.848419 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a366f33-a017-49d1-a2a6-8ee26b84e6b5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:51.848692 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a366f33-a017-49d1-a2a6-8ee26b84e6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:51.848960 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a366f33-a017-49d1-a2a6-8ee26b84e6b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:51.864320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a366f33-a017-49d1-a2a6-8ee26b84e6b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:51.867446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a366f33-a017-49d1-a2a6-8ee26b84e6b5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1a366f33-a017-49d1-a2a6-8ee26b84e6b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:35:51.911979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:51.912999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:35:51.913863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:35:51.913863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:51.913863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:51.914274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 168/343] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:51.921212 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cd33b-c109-49e3-af45-9e6ee8fe864e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:51.980710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:52.087241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cd33b-c109-49e3-af45-9e6ee8fe864e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:52.139089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5e989-73f6-4d9f-a549-9dbcbc3ff885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:52.139727 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5e989-73f6-4d9f-a549-9dbcbc3ff885 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:52.140038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5e989-73f6-4d9f-a549-9dbcbc3ff885 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:52.140312 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5e989-73f6-4d9f-a549-9dbcbc3ff885 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:52.140989 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5e989-73f6-4d9f-a549-9dbcbc3ff885 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:52.141284 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5e989-73f6-4d9f-a549-9dbcbc3ff885 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:52.142749 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5e989-73f6-4d9f-a549-9dbcbc3ff885 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:52.160526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 176/344] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:52.169659 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a366f33-a017-49d1-a2a6-8ee26b84e6b5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:52.221585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5e989-73f6-4d9f-a549-9dbcbc3ff885 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:52.334184 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:52.345493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a366f33-a017-49d1-a2a6-8ee26b84e6b5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:52.369970 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:52.387674 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0d443-a7f0-4fa7-9d8a-176226fee0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:52.387979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0d443-a7f0-4fa7-9d8a-176226fee0a5 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:52.388338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0d443-a7f0-4fa7-9d8a-176226fee0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:52.388484 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0d443-a7f0-4fa7-9d8a-176226fee0a5 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:52.389152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0d443-a7f0-4fa7-9d8a-176226fee0a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:52.390750 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0d443-a7f0-4fa7-9d8a-176226fee0a5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:52.399466 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5e989-73f6-4d9f-a549-9dbcbc3ff885 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:52.418591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 169/345] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:35:52 2022] POST /identity/v3/users => generated 246 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:35:52.424664 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0d443-a7f0-4fa7-9d8a-176226fee0a5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:52.479981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1da84f-9c47-4115-9b76-568654eab4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:52.481733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1da84f-9c47-4115-9b76-568654eab4ca None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:52.482563 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1da84f-9c47-4115-9b76-568654eab4ca None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:52.483410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1da84f-9c47-4115-9b76-568654eab4ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:52.514633 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1da84f-9c47-4115-9b76-568654eab4ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:52.518114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1da84f-9c47-4115-9b76-568654eab4ca None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-ce1da84f-9c47-4115-9b76-568654eab4ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:35:52.547167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:52.549255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:35:52.551165 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:35:52.551165 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:52.551165 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:52.554126 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 177/346] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:35:52.565903 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1da84f-9c47-4115-9b76-568654eab4ca None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:52.733768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0d443-a7f0-4fa7-9d8a-176226fee0a5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:52.762338 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1da84f-9c47-4115-9b76-568654eab4ca None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:52.808001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213f7b9-579e-4a09-ae94-eedc87c12071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:52.808470 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8715c1-86c4-4c9b-9b8a-6648e0a0b36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:52.808682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213f7b9-579e-4a09-ae94-eedc87c12071 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:52.808964 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213f7b9-579e-4a09-ae94-eedc87c12071 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:52.809237 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213f7b9-579e-4a09-ae94-eedc87c12071 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:52.810660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8715c1-86c4-4c9b-9b8a-6648e0a0b36b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:52.811487 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8715c1-86c4-4c9b-9b8a-6648e0a0b36b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:52.812313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8715c1-86c4-4c9b-9b8a-6648e0a0b36b None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:52.813765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8715c1-86c4-4c9b-9b8a-6648e0a0b36b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:52.814544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8715c1-86c4-4c9b-9b8a-6648e0a0b36b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:52.817519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8715c1-86c4-4c9b-9b8a-6648e0a0b36b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:52.822013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213f7b9-579e-4a09-ae94-eedc87c12071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:52.824119 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213f7b9-579e-4a09-ae94-eedc87c12071 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:52.831635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:52.842028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1213f7b9-579e-4a09-ae94-eedc87c12071 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:35:52.842282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:52.843952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:35:52.845650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:35:52.845650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:52.845650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:52.847211 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 170/347] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:52.850934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213f7b9-579e-4a09-ae94-eedc87c12071 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:52.857212 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 178/348] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:52.870532 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8715c1-86c4-4c9b-9b8a-6648e0a0b36b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:53.075680 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213f7b9-579e-4a09-ae94-eedc87c12071 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:53.151105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd682f-3bfa-4671-8b0b-c1cdf4846a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:53.152035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd682f-3bfa-4671-8b0b-c1cdf4846a24 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:53.152555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd682f-3bfa-4671-8b0b-c1cdf4846a24 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:53.153021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd682f-3bfa-4671-8b0b-c1cdf4846a24 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:53.153879 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd682f-3bfa-4671-8b0b-c1cdf4846a24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:53.154397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd682f-3bfa-4671-8b0b-c1cdf4846a24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:53.156097 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd682f-3bfa-4671-8b0b-c1cdf4846a24 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:53.175328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 171/349] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:53.177594 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd682f-3bfa-4671-8b0b-c1cdf4846a24 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:53.191698 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8715c1-86c4-4c9b-9b8a-6648e0a0b36b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:53.252561 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:53.269901 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b965b-6827-4bda-8577-481a8e7f659d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:53.270879 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b965b-6827-4bda-8577-481a8e7f659d None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:53.271317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b965b-6827-4bda-8577-481a8e7f659d None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:53.272118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b965b-6827-4bda-8577-481a8e7f659d None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:53.272956 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b965b-6827-4bda-8577-481a8e7f659d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:53.275728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b965b-6827-4bda-8577-481a8e7f659d None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:53.293175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd682f-3bfa-4671-8b0b-c1cdf4846a24 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:53.319907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 179/350] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:35:52 2022] POST /identity/v3/users => generated 248 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:35:53.319907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac019784-a3a0-4451-99c5-cde0c97ac82b None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:53.319907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac019784-a3a0-4451-99c5-cde0c97ac82b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:53.319907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac019784-a3a0-4451-99c5-cde0c97ac82b None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:53.324568 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 180/351] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:53.332370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d35dbf-6baa-495f-8b00-a94a453f1e90 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:53.332761 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d35dbf-6baa-495f-8b00-a94a453f1e90 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:53.333242 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d35dbf-6baa-495f-8b00-a94a453f1e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:53.377734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec835b-2b68-4462-a0f9-fc5ca57daf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:53.378793 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec835b-2b68-4462-a0f9-fc5ca57daf6c None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:53.379236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec835b-2b68-4462-a0f9-fc5ca57daf6c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:53.379719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec835b-2b68-4462-a0f9-fc5ca57daf6c None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:53.380908 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec835b-2b68-4462-a0f9-fc5ca57daf6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:53.381615 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-11d35dbf-6baa-495f-8b00-a94a453f1e90 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:53.383651 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec835b-2b68-4462-a0f9-fc5ca57daf6c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:53.383810 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-11d35dbf-6baa-495f-8b00-a94a453f1e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:53.457879 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 172/352] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:35:53 2022] POST /identity/v3/users => generated 248 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:35:53.494179 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:53.531037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 181/353] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:54.160687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec835b-2b68-4462-a0f9-fc5ca57daf6c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:54.290335 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96a6b4-a90c-44e0-b809-fa740950ea65 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:54.291908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96a6b4-a90c-44e0-b809-fa740950ea65 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:54.292675 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96a6b4-a90c-44e0-b809-fa740950ea65 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:54.295139 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 182/354] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:54.315449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db19f7e-3a0e-4a21-90bd-c4062897f7f9 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:54.316381 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db19f7e-3a0e-4a21-90bd-c4062897f7f9 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:54.317088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db19f7e-3a0e-4a21-90bd-c4062897f7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:54.376423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-7db19f7e-3a0e-4a21-90bd-c4062897f7f9 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:54.378588 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7db19f7e-3a0e-4a21-90bd-c4062897f7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:54.386241 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec835b-2b68-4462-a0f9-fc5ca57daf6c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:54.444483 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:54.498430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c2a5e-dbf5-46b8-b564-a664a209d487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:54.499104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c2a5e-dbf5-46b8-b564-a664a209d487 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:54.499380 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c2a5e-dbf5-46b8-b564-a664a209d487 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:54.499652 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c2a5e-dbf5-46b8-b564-a664a209d487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:54.520288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c2a5e-dbf5-46b8-b564-a664a209d487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:54.522944 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c2a5e-dbf5-46b8-b564-a664a209d487 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-362c2a5e-dbf5-46b8-b564-a664a209d487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:35:54.567936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:54.569734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:35:54.571417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:35:54.571417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:54.571417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:54.572478 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 183/355] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:54.574554 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 173/356] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:54.585462 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db19f7e-3a0e-4a21-90bd-c4062897f7f9 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:54.804994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db19f7e-3a0e-4a21-90bd-c4062897f7f9 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:54.842283 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:54.880641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30f71e-bfe5-4bf2-b3e4-86d6f2b8676a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:54.881684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30f71e-bfe5-4bf2-b3e4-86d6f2b8676a None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:54.882037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30f71e-bfe5-4bf2-b3e4-86d6f2b8676a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:54.882505 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30f71e-bfe5-4bf2-b3e4-86d6f2b8676a None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:54.883713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30f71e-bfe5-4bf2-b3e4-86d6f2b8676a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:54.884271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30f71e-bfe5-4bf2-b3e4-86d6f2b8676a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:54.886511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30f71e-bfe5-4bf2-b3e4-86d6f2b8676a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:54.907579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 184/357] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:54.913467 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c2a5e-dbf5-46b8-b564-a664a209d487 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:55.136240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30f71e-bfe5-4bf2-b3e4-86d6f2b8676a None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:55.144342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c2a5e-dbf5-46b8-b564-a664a209d487 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:55.259483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30f71e-bfe5-4bf2-b3e4-86d6f2b8676a None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:55.265191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f1389-495c-4263-bca0-b228e86a40bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:55.266872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f1389-495c-4263-bca0-b228e86a40bc None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:55.267956 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f1389-495c-4263-bca0-b228e86a40bc None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:55.268743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f1389-495c-4263-bca0-b228e86a40bc None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:55.270320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f1389-495c-4263-bca0-b228e86a40bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:55.273484 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f1389-495c-4263-bca0-b228e86a40bc None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:55.303766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e6a24-a6be-4148-b42d-c0062a60f2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:55.304899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e6a24-a6be-4148-b42d-c0062a60f2ab None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:55.304899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e6a24-a6be-4148-b42d-c0062a60f2ab None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:55.305145 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e6a24-a6be-4148-b42d-c0062a60f2ab None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:55.306008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e6a24-a6be-4148-b42d-c0062a60f2ab None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:55.308306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e6a24-a6be-4148-b42d-c0062a60f2ab None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:55.320373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 174/358] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:35:54 2022] POST /identity/v3/users => generated 249 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:35:55.333442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.sql.core [None req-f66e6a24-a6be-4148-b42d-c0062a60f2ab None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 15 13:35:55.333442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 15 13:35:55.333442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [parameters: {'id': '16f8678aec324f3d9c6ec183fd7e3bbb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 15 13:35:55.333442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87915) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-f66e6a24-a6be-4148-b42d-c0062a60f2ab None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application conn.query(q) Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application result.read() Oct 15 13:35:55.352496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return ref.to_dict() Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 13:35:55.354091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application next(self.gen) Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application yield resource Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application next(self.gen) Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise self.value Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application session.commit() Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 13:35:55.355827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self.session.flush() Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application compat.raise_( Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 13:35:55.357637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 13:35:55.359625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application conn.query(q) Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application result.read() Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 13:35:55.364952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [parameters: {'id': '16f8678aec324f3d9c6ec183fd7e3bbb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:55.372282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 15 13:35:55.375463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:55.382778 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 185/359] 10.222.0.36 () {60 vars in 1131 bytes} [Sat Oct 15 13:35:55 2022] POST /identity/v3/roles => generated 144 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:35:55.383014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f1389-495c-4263-bca0-b228e86a40bc None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:55.409824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:55.548822 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f1389-495c-4263-bca0-b228e86a40bc None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:55.589332 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f03c5fd-7208-4f08-917a-879605cbb4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:55.589988 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03c5fd-7208-4f08-917a-879605cbb4a7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:55.590259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03c5fd-7208-4f08-917a-879605cbb4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:55.590522 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03c5fd-7208-4f08-917a-879605cbb4a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:55.603568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f03c5fd-7208-4f08-917a-879605cbb4a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1f03c5fd-7208-4f08-917a-879605cbb4a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:55.609549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:55.612047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:55.613585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:55.613585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:55.613585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:55.613585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:55.613585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:55.613585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:55.613585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:55.613585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:55.613585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:55.613585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:55.613585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 15 13:35:55.613585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:55.613585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 175/360] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:35:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:55.620414 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e6a24-a6be-4148-b42d-c0062a60f2ab None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:55.668443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:55.741995 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e6a24-a6be-4148-b42d-c0062a60f2ab None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:55.784697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22289d4f-2961-4201-8c9b-16b66a13510a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:55.785809 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22289d4f-2961-4201-8c9b-16b66a13510a None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:55.786296 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22289d4f-2961-4201-8c9b-16b66a13510a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:55.786850 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22289d4f-2961-4201-8c9b-16b66a13510a None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:55.787780 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22289d4f-2961-4201-8c9b-16b66a13510a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:55.788333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22289d4f-2961-4201-8c9b-16b66a13510a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:55.790001 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22289d4f-2961-4201-8c9b-16b66a13510a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:55.800939 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 186/361] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:35:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:35:56.266030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e0b53-c7fb-4bcc-b846-d75f69b89255 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:56.266688 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e0b53-c7fb-4bcc-b846-d75f69b89255 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:56.267335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e0b53-c7fb-4bcc-b846-d75f69b89255 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:56.269572 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 176/362] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:56.275453 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b965075-bd07-425a-91bf-bd2f83e0d6fa None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:56.276099 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b965075-bd07-425a-91bf-bd2f83e0d6fa None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:56.276586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b965075-bd07-425a-91bf-bd2f83e0d6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:56.299303 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-0b965075-bd07-425a-91bf-bd2f83e0d6fa None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:56.300931 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0b965075-bd07-425a-91bf-bd2f83e0d6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:56.401660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 187/363] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:56.804596 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e0b53-c7fb-4bcc-b846-d75f69b89255 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:56.925053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:56.955701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e0b53-c7fb-4bcc-b846-d75f69b89255 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:57.002551 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd58446-4264-4693-8b96-165a7d1d5c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:57.002757 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7df55e-4342-4c7e-9166-12dde8f37f2d None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:57.003199 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7df55e-4342-4c7e-9166-12dde8f37f2d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:57.003670 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7df55e-4342-4c7e-9166-12dde8f37f2d None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:57.004128 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58446-4264-4693-8b96-165a7d1d5c17 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:57.004593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58446-4264-4693-8b96-165a7d1d5c17 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:57.005049 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58446-4264-4693-8b96-165a7d1d5c17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:57.006317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 188/364] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:57.013562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869fd04-b87c-4f97-9770-c0b7bfd14a03 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:57.014166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869fd04-b87c-4f97-9770-c0b7bfd14a03 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:57.014626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869fd04-b87c-4f97-9770-c0b7bfd14a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:57.033615 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd58446-4264-4693-8b96-165a7d1d5c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:57.035656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-8869fd04-b87c-4f97-9770-c0b7bfd14a03 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:57.037112 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8869fd04-b87c-4f97-9770-c0b7bfd14a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1cd58446-4264-4693-8b96-165a7d1d5c17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:57.043581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:57.044564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:57.045355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:57.045355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:57.045355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:57.045355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:57.045355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:57.045355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:57.045355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:57.045355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:57.045355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:57.045355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:57.045355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:35:57.045355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:57.052627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 177/365] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:35:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:57.061399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd58446-4264-4693-8b96-165a7d1d5c17 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:57.070239 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:57.121073 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 189/366] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:35:57.355038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee33c02-6c31-426e-b7a8-27f6ab703d86 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:57.355499 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee33c02-6c31-426e-b7a8-27f6ab703d86 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:57.356009 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee33c02-6c31-426e-b7a8-27f6ab703d86 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:57.356930 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd58446-4264-4693-8b96-165a7d1d5c17 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:57.358784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 190/367] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:57.367924 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cbfd1-28d9-4625-9597-39cb1ac4fcd4 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:57.368384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cbfd1-28d9-4625-9597-39cb1ac4fcd4 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:57.368823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cbfd1-28d9-4625-9597-39cb1ac4fcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:57.397256 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-7c7cbfd1-28d9-4625-9597-39cb1ac4fcd4 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:57.398874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7c7cbfd1-28d9-4625-9597-39cb1ac4fcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:57.470893 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae492d0-873f-43f3-ae46-b95cf0937e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:57.472015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae492d0-873f-43f3-ae46-b95cf0937e16 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:57.472495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae492d0-873f-43f3-ae46-b95cf0937e16 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:57.472966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae492d0-873f-43f3-ae46-b95cf0937e16 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:57.474407 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae492d0-873f-43f3-ae46-b95cf0937e16 None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:57.475315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae492d0-873f-43f3-ae46-b95cf0937e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:57.477784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae492d0-873f-43f3-ae46-b95cf0937e16 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:57.489393 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 191/368] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:57.491087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cbfd1-28d9-4625-9597-39cb1ac4fcd4 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:57.494591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 178/369] 10.222.0.36 () {56 vars in 1103 bytes} [Sat Oct 15 13:35:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:57.503026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae492d0-873f-43f3-ae46-b95cf0937e16 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:57.515045 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:57.780494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:57.808305 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cbfd1-28d9-4625-9597-39cb1ac4fcd4 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:57.818272 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae492d0-873f-43f3-ae46-b95cf0937e16 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:57.857454 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8999c-893a-4262-b997-1ebbd1c37890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:57.858107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8999c-893a-4262-b997-1ebbd1c37890 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:57.858382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8999c-893a-4262-b997-1ebbd1c37890 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:57.858650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8999c-893a-4262-b997-1ebbd1c37890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:57.876439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8999c-893a-4262-b997-1ebbd1c37890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:57.878174 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8999c-893a-4262-b997-1ebbd1c37890 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:57.888759 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2c9e8-9d43-43ff-993d-952b42375329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:57.889422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2c9e8-9d43-43ff-993d-952b42375329 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:57.889697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2c9e8-9d43-43ff-993d-952b42375329 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:57.889966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2c9e8-9d43-43ff-993d-952b42375329 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:57.904983 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2c9e8-9d43-43ff-993d-952b42375329 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-fad8999c-893a-4262-b997-1ebbd1c37890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:35:57.907112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:57.908768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:35:57.910363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:35:57.910363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:57.910363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-0aa2c9e8-9d43-43ff-993d-952b42375329 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:57.911661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:57.921052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:57.922691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:57.922691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:57.922691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:57.922691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:57.922691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:57.922691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:57.922691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:57.922691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:57.922691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:57.922691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:57.922691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:35:57.922691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:57.924491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 179/371] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:57.924619 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 192/371] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:35:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:57.924744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8999c-893a-4262-b997-1ebbd1c37890 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:57.928977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2c9e8-9d43-43ff-993d-952b42375329 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:58.035849 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:58.064422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2c9e8-9d43-43ff-993d-952b42375329 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:58.102243 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a011896-e6de-4a2a-8d4d-7f71d80f9685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:58.102867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a011896-e6de-4a2a-8d4d-7f71d80f9685 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:58.103146 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a011896-e6de-4a2a-8d4d-7f71d80f9685 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:58.103418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a011896-e6de-4a2a-8d4d-7f71d80f9685 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:58.104105 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a011896-e6de-4a2a-8d4d-7f71d80f9685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:58.104406 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a011896-e6de-4a2a-8d4d-7f71d80f9685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:58.105800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a011896-e6de-4a2a-8d4d-7f71d80f9685 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:58.118328 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 193/372] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:58.119287 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a011896-e6de-4a2a-8d4d-7f71d80f9685 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:58.133318 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:58.137446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8999c-893a-4262-b997-1ebbd1c37890 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:58.202270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad2e3f-e065-4a13-aa9d-cd61016d5003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:58.203351 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad2e3f-e065-4a13-aa9d-cd61016d5003 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:58.203836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad2e3f-e065-4a13-aa9d-cd61016d5003 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:58.204417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad2e3f-e065-4a13-aa9d-cd61016d5003 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:58.224074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad2e3f-e065-4a13-aa9d-cd61016d5003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:58.225008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a011896-e6de-4a2a-8d4d-7f71d80f9685 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-5cad2e3f-e065-4a13-aa9d-cd61016d5003 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:58.234696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:58.235865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:58.237994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:58.237994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:58.237994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:58.237994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:58.237994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:58.237994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:58.237994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:58.237994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:58.237994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:58.237994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:58.237994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:35:58.237994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:58.237994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 180/373] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:35:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:58.239645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad2e3f-e065-4a13-aa9d-cd61016d5003 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:58.276717 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc85b7d-f316-448f-86be-67ae7a9168dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:58.277681 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc85b7d-f316-448f-86be-67ae7a9168dc None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:58.278142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc85b7d-f316-448f-86be-67ae7a9168dc None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:58.278805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc85b7d-f316-448f-86be-67ae7a9168dc None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:58.280334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc85b7d-f316-448f-86be-67ae7a9168dc None admin] RBAC: Adding query filter params (name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:58.280922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc85b7d-f316-448f-86be-67ae7a9168dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:58.284260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc85b7d-f316-448f-86be-67ae7a9168dc None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:58.302277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 194/374] 10.222.0.36 () {56 vars in 1103 bytes} [Sat Oct 15 13:35:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:58.303719 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc85b7d-f316-448f-86be-67ae7a9168dc None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:58.360580 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad2e3f-e065-4a13-aa9d-cd61016d5003 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:58.379914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:58.402406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff950e3f-b75e-476e-bb60-e4b7723f69cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:58.403451 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff950e3f-b75e-476e-bb60-e4b7723f69cc None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:58.403968 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff950e3f-b75e-476e-bb60-e4b7723f69cc None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:58.404437 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff950e3f-b75e-476e-bb60-e4b7723f69cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:58.414005 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff950e3f-b75e-476e-bb60-e4b7723f69cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:58.416053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff950e3f-b75e-476e-bb60-e4b7723f69cc None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-ff950e3f-b75e-476e-bb60-e4b7723f69cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:35:58.430783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:58.431947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:58.432813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:58.432813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:58.432813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:35:58.432813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:35:58.432813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:35:58.432813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:35:58.432813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:35:58.432813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:35:58.432813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 13:35:58.432813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:58.436293 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 181/375] 10.222.0.36 () {56 vars in 1093 bytes} [Sat Oct 15 13:35:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:58.437576 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff950e3f-b75e-476e-bb60-e4b7723f69cc None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:58.462569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:58.514170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc85b7d-f316-448f-86be-67ae7a9168dc None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:58.564075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d2969-7f66-4c68-b889-3930e435a242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:58.565992 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d2969-7f66-4c68-b889-3930e435a242 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:58.566473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d2969-7f66-4c68-b889-3930e435a242 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:58.566918 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d2969-7f66-4c68-b889-3930e435a242 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:58.568141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d2969-7f66-4c68-b889-3930e435a242 None admin] RBAC: Adding query filter params (name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:58.568632 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d2969-7f66-4c68-b889-3930e435a242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:58.570556 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d2969-7f66-4c68-b889-3930e435a242 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:58.587192 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 195/376] 10.222.0.36 () {56 vars in 1103 bytes} [Sat Oct 15 13:35:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:58.587824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d2969-7f66-4c68-b889-3930e435a242 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:58.622298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff950e3f-b75e-476e-bb60-e4b7723f69cc None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:58.661942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc1af4-d5a3-411c-843a-a14eb0f722fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:58.663005 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc1af4-d5a3-411c-843a-a14eb0f722fa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:58.663475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc1af4-d5a3-411c-843a-a14eb0f722fa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:58.663986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc1af4-d5a3-411c-843a-a14eb0f722fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:58.677256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc1af4-d5a3-411c-843a-a14eb0f722fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:58.679157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc1af4-d5a3-411c-843a-a14eb0f722fa None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-cdcc1af4-d5a3-411c-843a-a14eb0f722fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:35:58.696471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:58.697421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:35:58.698258 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:35:58.698258 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:58.698258 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:58.700545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 182/377] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:58.701708 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc1af4-d5a3-411c-843a-a14eb0f722fa None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:58.735398 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:58.798266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d2969-7f66-4c68-b889-3930e435a242 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:58.861802 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc1af4-d5a3-411c-843a-a14eb0f722fa None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:58.869356 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3032d-49b0-4eb8-9587-bc05f6a535d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:58.870610 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3032d-49b0-4eb8-9587-bc05f6a535d9 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:58.871142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3032d-49b0-4eb8-9587-bc05f6a535d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:58.871664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3032d-49b0-4eb8-9587-bc05f6a535d9 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:58.873043 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3032d-49b0-4eb8-9587-bc05f6a535d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:58.873619 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3032d-49b0-4eb8-9587-bc05f6a535d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:58.876194 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3032d-49b0-4eb8-9587-bc05f6a535d9 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:58.881300 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:58.936917 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b0cab-ec40-47b2-bc7e-173829654957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:58.939822 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b0cab-ec40-47b2-bc7e-173829654957 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:58.940266 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b0cab-ec40-47b2-bc7e-173829654957 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:58.940672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b0cab-ec40-47b2-bc7e-173829654957 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:58.961430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 196/378] 10.222.0.36 () {56 vars in 1133 bytes} [Sat Oct 15 13:35:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:58.962198 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b0cab-ec40-47b2-bc7e-173829654957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:58.963188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3032d-49b0-4eb8-9587-bc05f6a535d9 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:58.966126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b0cab-ec40-47b2-bc7e-173829654957 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-a17b0cab-ec40-47b2-bc7e-173829654957 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:35:58.995967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:58.997846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:35:58.999616 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:35:58.999616 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:58.999616 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:58.999616 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 183/379] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:59.001366 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d472bac-4420-42a4-9f12-ac6dd9497c57 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:59.001800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d472bac-4420-42a4-9f12-ac6dd9497c57 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:59.002224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d472bac-4420-42a4-9f12-ac6dd9497c57 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:59.005530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 184/380] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:59.005828 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d472bac-4420-42a4-9f12-ac6dd9497c57 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:59.103275 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3032d-49b0-4eb8-9587-bc05f6a535d9 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:59.110975 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:59.135992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d472bac-4420-42a4-9f12-ac6dd9497c57 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:59.147778 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29910718-423d-4b3f-82e0-753f9dd8a27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:59.148728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29910718-423d-4b3f-82e0-753f9dd8a27f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:59.149052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29910718-423d-4b3f-82e0-753f9dd8a27f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:59.149484 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29910718-423d-4b3f-82e0-753f9dd8a27f None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:59.150536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29910718-423d-4b3f-82e0-753f9dd8a27f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:59.151021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29910718-423d-4b3f-82e0-753f9dd8a27f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:59.153325 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29910718-423d-4b3f-82e0-753f9dd8a27f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:59.171927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 197/381] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:59.174091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d4e75-3687-4caf-ab31-8d0fc431515f None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:59.174546 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d4e75-3687-4caf-ab31-8d0fc431515f None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:59.174888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d4e75-3687-4caf-ab31-8d0fc431515f None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:59.176309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 198/382] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:35:59.177778 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d4e75-3687-4caf-ab31-8d0fc431515f None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:59.195583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256e2ec-266a-4fb2-8f8e-642983808603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:59.196445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256e2ec-266a-4fb2-8f8e-642983808603 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:59.196854 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256e2ec-266a-4fb2-8f8e-642983808603 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:59.197222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256e2ec-266a-4fb2-8f8e-642983808603 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:59.211620 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256e2ec-266a-4fb2-8f8e-642983808603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:59.213461 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256e2ec-266a-4fb2-8f8e-642983808603 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-6256e2ec-266a-4fb2-8f8e-642983808603 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:35:59.231360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:59.233350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:35:59.235291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:35:59.235291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:35:59.235291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:59.235291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 185/383] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:59.236192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576451e4-9023-422f-a901-9eff63407727 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:59.236653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576451e4-9023-422f-a901-9eff63407727 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:59.237082 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576451e4-9023-422f-a901-9eff63407727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:59.247634 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:59.262228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-576451e4-9023-422f-a901-9eff63407727 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:59.263531 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-576451e4-9023-422f-a901-9eff63407727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:59.316035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d4e75-3687-4caf-ab31-8d0fc431515f None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:59.367914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 186/384] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:59.367914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576451e4-9023-422f-a901-9eff63407727 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:59.368247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c3f38-53bd-4af2-8a06-9ce81a9edc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:59.368247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c3f38-53bd-4af2-8a06-9ce81a9edc65 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:59.368247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c3f38-53bd-4af2-8a06-9ce81a9edc65 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:59.368247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c3f38-53bd-4af2-8a06-9ce81a9edc65 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:59.369472 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c3f38-53bd-4af2-8a06-9ce81a9edc65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:59.370007 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c3f38-53bd-4af2-8a06-9ce81a9edc65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:59.373185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c3f38-53bd-4af2-8a06-9ce81a9edc65 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:59.395435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 199/385] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:59.397625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19725fd-da03-45f2-9363-c7e74a597f80 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:59.398118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19725fd-da03-45f2-9363-c7e74a597f80 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:59.398564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19725fd-da03-45f2-9363-c7e74a597f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:59.426767 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-d19725fd-da03-45f2-9363-c7e74a597f80 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:35:59.428563 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-d19725fd-da03-45f2-9363-c7e74a597f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:35:59.445816 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:59.504548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576451e4-9023-422f-a901-9eff63407727 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:59.538055 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 200/386] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:35:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:35:59.538055 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19725fd-da03-45f2-9363-c7e74a597f80 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:59.575367 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:59.581134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a75540-32c0-423c-aa50-e27c7e17f8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:59.584080 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a75540-32c0-423c-aa50-e27c7e17f8e9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:59.584593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a75540-32c0-423c-aa50-e27c7e17f8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:59.585768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a75540-32c0-423c-aa50-e27c7e17f8e9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:59.602293 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a75540-32c0-423c-aa50-e27c7e17f8e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:59.605764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a75540-32c0-423c-aa50-e27c7e17f8e9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f2a75540-32c0-423c-aa50-e27c7e17f8e9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:35:59.631952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:59.633843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:59.635827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:59.635827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:59.635827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:35:59.635827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:35:59.635827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:35:59.635827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:35:59.635827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:35:59.635827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:35:59.635827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 15 13:35:59.635827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:59.640899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 187/387] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:59.642697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a75540-32c0-423c-aa50-e27c7e17f8e9 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:59.648923 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19725fd-da03-45f2-9363-c7e74a597f80 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:59.689948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd64797-7508-4d1a-aeb8-64b13957cd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:59.690587 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd64797-7508-4d1a-aeb8-64b13957cd19 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:59.690950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd64797-7508-4d1a-aeb8-64b13957cd19 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:59.691248 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd64797-7508-4d1a-aeb8-64b13957cd19 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:59.691932 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd64797-7508-4d1a-aeb8-64b13957cd19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:35:59.692244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd64797-7508-4d1a-aeb8-64b13957cd19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:59.693663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd64797-7508-4d1a-aeb8-64b13957cd19 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:59.714515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 201/388] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:35:59.714515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd64797-7508-4d1a-aeb8-64b13957cd19 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:59.834962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a75540-32c0-423c-aa50-e27c7e17f8e9 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:59.839073 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd64797-7508-4d1a-aeb8-64b13957cd19 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:35:59.842129 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:59.870431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:35:59.895206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85525b-529c-45fe-b77b-2910705da6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:59.896187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85525b-529c-45fe-b77b-2910705da6d4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:59.896664 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85525b-529c-45fe-b77b-2910705da6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:59.897134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85525b-529c-45fe-b77b-2910705da6d4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:59.898772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329867a8-4944-4835-979c-4156b33a23d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:35:59.899508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329867a8-4944-4835-979c-4156b33a23d8 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:35:59.899703 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329867a8-4944-4835-979c-4156b33a23d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:35:59.900008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329867a8-4944-4835-979c-4156b33a23d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:35:59.911037 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85525b-529c-45fe-b77b-2910705da6d4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:35:59.911348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329867a8-4944-4835-979c-4156b33a23d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:35:59.913738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85525b-529c-45fe-b77b-2910705da6d4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-329867a8-4944-4835-979c-4156b33a23d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:35:59.916412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:59.920449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:59.921988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:59.921988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:59.921988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:59.921988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:59.921988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:35:59.921988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:35:59.921988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:35:59.921988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:35:59.921988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:35:59.921988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:35:59.921988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:35:59.921988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:35:59.921988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 202/389] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:35:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:59.921988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329867a8-4944-4835-979c-4156b33a23d8 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1d85525b-529c-45fe-b77b-2910705da6d4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:35:59.936591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:35:59.937769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:35:59.939627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:35:59.939627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:35:59.939627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:35:59.939627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:35:59.939627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:35:59.939627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:35:59.939627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:35:59.939627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:35:59.939627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 13:35:59.939627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:35:59.939627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 188/390] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:35:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:35:59.940735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85525b-529c-45fe-b77b-2910705da6d4 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:00.060800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85525b-529c-45fe-b77b-2910705da6d4 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:00.078687 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329867a8-4944-4835-979c-4156b33a23d8 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:00.100675 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad97ed-01c7-46ee-b9a8-672d46248d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:00.101316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad97ed-01c7-46ee-b9a8-672d46248d45 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:00.101604 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad97ed-01c7-46ee-b9a8-672d46248d45 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:00.101871 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad97ed-01c7-46ee-b9a8-672d46248d45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:00.110005 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad97ed-01c7-46ee-b9a8-672d46248d45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:00.111767 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad97ed-01c7-46ee-b9a8-672d46248d45 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-83ad97ed-01c7-46ee-b9a8-672d46248d45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:00.124647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:00.126399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:00.128043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:00.128043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:00.128043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:00.128043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:00.128043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:00.128043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:00.128043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:00.128043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:00.128043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 189/391] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:35:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:00.128043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad97ed-01c7-46ee-b9a8-672d46248d45 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:00.144688 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d398748-5a0f-41b8-b6b4-bda6a6bc4492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:00.146118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d398748-5a0f-41b8-b6b4-bda6a6bc4492 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:00.146618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d398748-5a0f-41b8-b6b4-bda6a6bc4492 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:00.147120 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d398748-5a0f-41b8-b6b4-bda6a6bc4492 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:00.148535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d398748-5a0f-41b8-b6b4-bda6a6bc4492 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:00.149094 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d398748-5a0f-41b8-b6b4-bda6a6bc4492 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:00.152007 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d398748-5a0f-41b8-b6b4-bda6a6bc4492 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:00.179588 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 203/392] 10.222.0.36 () {56 vars in 1137 bytes} [Sat Oct 15 13:35:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:00.180342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d398748-5a0f-41b8-b6b4-bda6a6bc4492 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:00.193511 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:00.209715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:00.276761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad97ed-01c7-46ee-b9a8-672d46248d45 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:00.315156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cf259-e4c7-41ec-ae63-5a2779ccc9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:00.315417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d398748-5a0f-41b8-b6b4-bda6a6bc4492 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:00.315807 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cf259-e4c7-41ec-ae63-5a2779ccc9c8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:00.316109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cf259-e4c7-41ec-ae63-5a2779ccc9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:00.316379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cf259-e4c7-41ec-ae63-5a2779ccc9c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:00.329171 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cf259-e4c7-41ec-ae63-5a2779ccc9c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:00.331577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cf259-e4c7-41ec-ae63-5a2779ccc9c8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-913cf259-e4c7-41ec-ae63-5a2779ccc9c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:00.348623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:00.350711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:00.353330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:00.353330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:00.353330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:00.353330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 190/393] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:00.353330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cf259-e4c7-41ec-ae63-5a2779ccc9c8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:00.373308 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddcee7-2114-4746-8795-782a1c5064f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:00.373934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddcee7-2114-4746-8795-782a1c5064f2 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:00.374206 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddcee7-2114-4746-8795-782a1c5064f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:00.374473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddcee7-2114-4746-8795-782a1c5064f2 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:00.375140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecddcee7-2114-4746-8795-782a1c5064f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:00.375441 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecddcee7-2114-4746-8795-782a1c5064f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:00.377560 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecddcee7-2114-4746-8795-782a1c5064f2 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:00.386133 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 204/394] 10.222.0.36 () {56 vars in 1103 bytes} [Sat Oct 15 13:36:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:00.387000 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddcee7-2114-4746-8795-782a1c5064f2 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:00.482017 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:00.491346 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cf259-e4c7-41ec-ae63-5a2779ccc9c8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:00.504490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:00.527113 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddcee7-2114-4746-8795-782a1c5064f2 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:00.531388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7f979-8427-45a0-933f-23c2cef98d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:00.532090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7f979-8427-45a0-933f-23c2cef98d3a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:00.532375 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7f979-8427-45a0-933f-23c2cef98d3a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:00.532799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7f979-8427-45a0-933f-23c2cef98d3a None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:00.533553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7f979-8427-45a0-933f-23c2cef98d3a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:00.533857 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7f979-8427-45a0-933f-23c2cef98d3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:00.535389 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7f979-8427-45a0-933f-23c2cef98d3a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:00.571715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2d6e4-9c0d-494e-93a7-0e9d1b5b354c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:00.572699 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2d6e4-9c0d-494e-93a7-0e9d1b5b354c None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:00.573079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2d6e4-9c0d-494e-93a7-0e9d1b5b354c None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:00.573435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2d6e4-9c0d-494e-93a7-0e9d1b5b354c None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:00.574242 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2d6e4-9c0d-494e-93a7-0e9d1b5b354c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:00.574641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2d6e4-9c0d-494e-93a7-0e9d1b5b354c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:00.576217 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2d6e4-9c0d-494e-93a7-0e9d1b5b354c None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:00.586054 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 191/395] 10.222.0.36 () {56 vars in 1137 bytes} [Sat Oct 15 13:36:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:00.587474 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7f979-8427-45a0-933f-23c2cef98d3a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:00.601745 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 205/396] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:00.601745 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2d6e4-9c0d-494e-93a7-0e9d1b5b354c None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:00.736829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7f979-8427-45a0-933f-23c2cef98d3a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:00.753163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2d6e4-9c0d-494e-93a7-0e9d1b5b354c None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:00.780998 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec5412-10a0-4626-af20-7b867b830fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:00.781818 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec5412-10a0-4626-af20-7b867b830fb5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:00.782215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec5412-10a0-4626-af20-7b867b830fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:00.782938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec5412-10a0-4626-af20-7b867b830fb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:00.795048 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec5412-10a0-4626-af20-7b867b830fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:00.795553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b56410-c38f-4f4f-b34f-18fbe7c88560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:00.796327 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b56410-c38f-4f4f-b34f-18fbe7c88560 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:00.796684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b56410-c38f-4f4f-b34f-18fbe7c88560 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:00.797013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec5412-10a0-4626-af20-7b867b830fb5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:00.797169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b56410-c38f-4f4f-b34f-18fbe7c88560 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:00.797791 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b56410-c38f-4f4f-b34f-18fbe7c88560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:00.798260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b56410-c38f-4f4f-b34f-18fbe7c88560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:00.800214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b56410-c38f-4f4f-b34f-18fbe7c88560 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:00.814012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 206/397] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-a8ec5412-10a0-4626-af20-7b867b830fb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:00.815294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:00.816983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:00.818712 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:00.818712 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:00.818712 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:00.819016 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b56410-c38f-4f4f-b34f-18fbe7c88560 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:00.819140 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 192/398] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:00.819140 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec5412-10a0-4626-af20-7b867b830fb5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:00.823089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:00.827047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:00.962214 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec5412-10a0-4626-af20-7b867b830fb5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:00.965081 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b56410-c38f-4f4f-b34f-18fbe7c88560 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:01.000856 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90af48-b14a-40d5-9426-2de99c02b0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:01.001653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90af48-b14a-40d5-9426-2de99c02b0fd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:01.002058 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90af48-b14a-40d5-9426-2de99c02b0fd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:01.002429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90af48-b14a-40d5-9426-2de99c02b0fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:01.004062 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0752b5-4a1c-425d-9824-aedc81a380fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:01.004721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0752b5-4a1c-425d-9824-aedc81a380fa None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:01.005001 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0752b5-4a1c-425d-9824-aedc81a380fa None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:01.005266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0752b5-4a1c-425d-9824-aedc81a380fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:01.016273 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90af48-b14a-40d5-9426-2de99c02b0fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:01.016682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0752b5-4a1c-425d-9824-aedc81a380fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:01.018148 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90af48-b14a-40d5-9426-2de99c02b0fd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:01.018480 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0752b5-4a1c-425d-9824-aedc81a380fa None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-1b0752b5-4a1c-425d-9824-aedc81a380fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:01.036153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:01.038183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:01.040197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:01.040197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:01.040197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:01.040197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 207/399] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-3f90af48-b14a-40d5-9426-2de99c02b0fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:01.040679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:01.043112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:01.044155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:01.044155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:01.044155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:01.044155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 193/400] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:01.044155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90af48-b14a-40d5-9426-2de99c02b0fd None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:01.044446 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0752b5-4a1c-425d-9824-aedc81a380fa None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:01.106196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:01.124468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:01.176165 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0752b5-4a1c-425d-9824-aedc81a380fa None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:01.192112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90af48-b14a-40d5-9426-2de99c02b0fd None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:01.216725 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15ba4c-11ba-4575-ae0e-e7c537391591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:01.217692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15ba4c-11ba-4575-ae0e-e7c537391591 None admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:01.218084 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15ba4c-11ba-4575-ae0e-e7c537391591 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:01.218398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15ba4c-11ba-4575-ae0e-e7c537391591 None admin] PATH_INFO: `/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/2e13692c1e83413a909d99bb7db2e4b8/roles/8b837ae2ed57479a93acb638b80e2d67` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:01.235454 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340eab32-afc7-48f8-84bc-b2bef249e649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:01.237006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eab32-afc7-48f8-84bc-b2bef249e649 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:01.237449 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eab32-afc7-48f8-84bc-b2bef249e649 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:01.237876 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eab32-afc7-48f8-84bc-b2bef249e649 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:01.238034 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15ba4c-11ba-4575-ae0e-e7c537391591 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a5a64cc8aa04972bb464407cb0312b4, user_id=2e13692c1e83413a909d99bb7db2e4b8, role_id=8b837ae2ed57479a93acb638b80e2d67)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:01.238954 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340eab32-afc7-48f8-84bc-b2bef249e649 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:01.240660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15ba4c-11ba-4575-ae0e-e7c537391591 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:01.241427 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340eab32-afc7-48f8-84bc-b2bef249e649 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:01.261012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 208/401] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:36:01 2022] PUT /identity/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/2e13692c1e83413a909d99bb7db2e4b8/roles/8b837ae2ed57479a93acb638b80e2d67 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:36:01.262415 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15ba4c-11ba-4575-ae0e-e7c537391591 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:01.269081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 194/402] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:36:01 2022] POST /identity/v3/users => generated 247 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:36:01.270508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340eab32-afc7-48f8-84bc-b2bef249e649 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:01.451224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340eab32-afc7-48f8-84bc-b2bef249e649 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:01.463148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15ba4c-11ba-4575-ae0e-e7c537391591 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:01.467351 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:01.479667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:01.518736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14815001-57d5-4f8a-b33c-f8bd454ce3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:01.519447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14815001-57d5-4f8a-b33c-f8bd454ce3b3 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:01.520253 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14815001-57d5-4f8a-b33c-f8bd454ce3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:01.520600 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14815001-57d5-4f8a-b33c-f8bd454ce3b3 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:01.521223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14815001-57d5-4f8a-b33c-f8bd454ce3b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:01.521527 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14815001-57d5-4f8a-b33c-f8bd454ce3b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:01.523991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14815001-57d5-4f8a-b33c-f8bd454ce3b3 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:01.531180 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c600cb6-7e43-40db-9973-5215541f8ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:01.533495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c600cb6-7e43-40db-9973-5215541f8ac8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:01.534394 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c600cb6-7e43-40db-9973-5215541f8ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:01.535126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c600cb6-7e43-40db-9973-5215541f8ac8 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:01.536830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c600cb6-7e43-40db-9973-5215541f8ac8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:01.537710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c600cb6-7e43-40db-9973-5215541f8ac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:01.538277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 209/403] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:01.539172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14815001-57d5-4f8a-b33c-f8bd454ce3b3 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:01.539797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c600cb6-7e43-40db-9973-5215541f8ac8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:01.554857 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 195/404] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:01.555835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c600cb6-7e43-40db-9973-5215541f8ac8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:01.671923 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14815001-57d5-4f8a-b33c-f8bd454ce3b3 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:01.678013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c600cb6-7e43-40db-9973-5215541f8ac8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:01.710866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002023d3-cc33-4d1a-b3fa-7770e9f09604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:01.711604 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002023d3-cc33-4d1a-b3fa-7770e9f09604 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:01.711994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002023d3-cc33-4d1a-b3fa-7770e9f09604 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:01.712173 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002023d3-cc33-4d1a-b3fa-7770e9f09604 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:01.712843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002023d3-cc33-4d1a-b3fa-7770e9f09604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:01.713344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002023d3-cc33-4d1a-b3fa-7770e9f09604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:01.714435 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea42e1-3654-43f3-a7b9-d6e74f4bc1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:01.714757 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002023d3-cc33-4d1a-b3fa-7770e9f09604 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:01.715077 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea42e1-3654-43f3-a7b9-d6e74f4bc1ff None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:01.715348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea42e1-3654-43f3-a7b9-d6e74f4bc1ff None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:01.715593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea42e1-3654-43f3-a7b9-d6e74f4bc1ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:01.723740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea42e1-3654-43f3-a7b9-d6e74f4bc1ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:01.725369 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea42e1-3654-43f3-a7b9-d6e74f4bc1ff None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:01.729066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 210/405] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:01.729966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002023d3-cc33-4d1a-b3fa-7770e9f09604 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-b2ea42e1-3654-43f3-a7b9-d6e74f4bc1ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:01.745069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:01.746736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:01.748417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:01.748417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:01.748417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:01.748417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:01.748417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:01.748417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:01.748417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:01.748417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:01.752567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 196/406] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:01.754521 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea42e1-3654-43f3-a7b9-d6e74f4bc1ff None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:01.759454 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:01.841102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:01.856902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002023d3-cc33-4d1a-b3fa-7770e9f09604 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:01.913024 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91007c2b-441a-47d1-98a6-bf6e3561259b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:01.914641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91007c2b-441a-47d1-98a6-bf6e3561259b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:01.915071 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91007c2b-441a-47d1-98a6-bf6e3561259b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:01.915557 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91007c2b-441a-47d1-98a6-bf6e3561259b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:01.929027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91007c2b-441a-47d1-98a6-bf6e3561259b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:01.930510 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91007c2b-441a-47d1-98a6-bf6e3561259b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-91007c2b-441a-47d1-98a6-bf6e3561259b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:01.946609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:01.947562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:01.948471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:01.948471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:01.948471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:01.948471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:01.948471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:01.948471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:01.948471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:01.948471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:01.948471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 13:36:01.948471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:01.952301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 211/407] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:36:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:01.953260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91007c2b-441a-47d1-98a6-bf6e3561259b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:01.961357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea42e1-3654-43f3-a7b9-d6e74f4bc1ff None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:02.001586 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d6488-b61b-4df0-a2d7-c05583f7fb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:02.002734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d6488-b61b-4df0-a2d7-c05583f7fb84 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:02.002931 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d6488-b61b-4df0-a2d7-c05583f7fb84 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:02.003972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d6488-b61b-4df0-a2d7-c05583f7fb84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:02.016118 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082d6488-b61b-4df0-a2d7-c05583f7fb84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:02.019490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082d6488-b61b-4df0-a2d7-c05583f7fb84 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-082d6488-b61b-4df0-a2d7-c05583f7fb84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:02.036357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:02.037574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:02.038710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:02.038710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:02.038710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:02.038710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:02.038710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:02.038710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:02.038710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:02.038710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:02.040708 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 197/408] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:02.042702 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d6488-b61b-4df0-a2d7-c05583f7fb84 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:02.060335 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:02.081318 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91007c2b-441a-47d1-98a6-bf6e3561259b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:02.129000 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd4006-0c1b-4eed-b01e-ea85d0189960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:02.129638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd4006-0c1b-4eed-b01e-ea85d0189960 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:02.129914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd4006-0c1b-4eed-b01e-ea85d0189960 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:02.130183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd4006-0c1b-4eed-b01e-ea85d0189960 None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:02.130890 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd4006-0c1b-4eed-b01e-ea85d0189960 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:02.131195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd4006-0c1b-4eed-b01e-ea85d0189960 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:02.132755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd4006-0c1b-4eed-b01e-ea85d0189960 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:02.151920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 212/409] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:02.151920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd4006-0c1b-4eed-b01e-ea85d0189960 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:02.211268 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:02.225236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d6488-b61b-4df0-a2d7-c05583f7fb84 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:02.267990 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92de49f-cac7-4ade-aa7b-6b0c655d0752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:02.268338 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd4006-0c1b-4eed-b01e-ea85d0189960 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:02.269083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92de49f-cac7-4ade-aa7b-6b0c655d0752 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:02.269554 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92de49f-cac7-4ade-aa7b-6b0c655d0752 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:02.269965 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92de49f-cac7-4ade-aa7b-6b0c655d0752 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:02.270971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92de49f-cac7-4ade-aa7b-6b0c655d0752 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:02.271529 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92de49f-cac7-4ade-aa7b-6b0c655d0752 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:02.273282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92de49f-cac7-4ade-aa7b-6b0c655d0752 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:02.300425 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 198/410] 10.222.0.36 () {56 vars in 1139 bytes} [Sat Oct 15 13:36:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:02.301785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92de49f-cac7-4ade-aa7b-6b0c655d0752 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:02.307823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1da8f5-7a29-4b66-81ee-1fe45fdeefe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:02.308194 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da8f5-7a29-4b66-81ee-1fe45fdeefe5 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:02.308451 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da8f5-7a29-4b66-81ee-1fe45fdeefe5 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:02.308724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da8f5-7a29-4b66-81ee-1fe45fdeefe5 None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:02.309445 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1da8f5-7a29-4b66-81ee-1fe45fdeefe5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:02.309750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1da8f5-7a29-4b66-81ee-1fe45fdeefe5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:02.311252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1da8f5-7a29-4b66-81ee-1fe45fdeefe5 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:02.325148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 213/411] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:02.326136 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1da8f5-7a29-4b66-81ee-1fe45fdeefe5 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:02.330806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:02.435817 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92de49f-cac7-4ade-aa7b-6b0c655d0752 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:02.450185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1da8f5-7a29-4b66-81ee-1fe45fdeefe5 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:02.479292 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4d4f6-92bb-4bcb-8076-a494004ed724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:02.480564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d4f6-92bb-4bcb-8076-a494004ed724 None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:02.481190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d4f6-92bb-4bcb-8076-a494004ed724 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:02.481667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d4f6-92bb-4bcb-8076-a494004ed724 None admin] PATH_INFO: `/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/4f5aabcb1414413a8dce7b76ce701582/roles/8b837ae2ed57479a93acb638b80e2d67` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:02.490647 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85c1a1-8db4-463d-9a54-4eb6be79fd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:02.491323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85c1a1-8db4-463d-9a54-4eb6be79fd92 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:02.491598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85c1a1-8db4-463d-9a54-4eb6be79fd92 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:02.491904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85c1a1-8db4-463d-9a54-4eb6be79fd92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:02.502489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4d4f6-92bb-4bcb-8076-a494004ed724 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a5a64cc8aa04972bb464407cb0312b4, user_id=4f5aabcb1414413a8dce7b76ce701582, role_id=8b837ae2ed57479a93acb638b80e2d67)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:02.504426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85c1a1-8db4-463d-9a54-4eb6be79fd92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:02.504716 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4d4f6-92bb-4bcb-8076-a494004ed724 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:02.506182 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85c1a1-8db4-463d-9a54-4eb6be79fd92 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:02.519413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-5a85c1a1-8db4-463d-9a54-4eb6be79fd92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:02.524544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:02.525607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:02.526455 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:02.526455 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:02.526455 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:02.526455 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 214/412] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:02.531073 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85c1a1-8db4-463d-9a54-4eb6be79fd92 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:02.532315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 199/413] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:36:02 2022] PUT /identity/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/4f5aabcb1414413a8dce7b76ce701582/roles/8b837ae2ed57479a93acb638b80e2d67 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:36:02.542800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4d4f6-92bb-4bcb-8076-a494004ed724 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:02.668290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:02.774137 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4d4f6-92bb-4bcb-8076-a494004ed724 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:02.775290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85c1a1-8db4-463d-9a54-4eb6be79fd92 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:02.822896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1dc1f3-b2c4-4bd8-b2a8-c9995a0451c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:02.823957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1dc1f3-b2c4-4bd8-b2a8-c9995a0451c2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:02.824467 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1dc1f3-b2c4-4bd8-b2a8-c9995a0451c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:02.824934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1dc1f3-b2c4-4bd8-b2a8-c9995a0451c2 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:02.825786 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1dc1f3-b2c4-4bd8-b2a8-c9995a0451c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:02.826293 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1dc1f3-b2c4-4bd8-b2a8-c9995a0451c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:02.827957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1dc1f3-b2c4-4bd8-b2a8-c9995a0451c2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:02.837425 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6665d-3257-4712-bc69-f983470d44ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:02.838119 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6665d-3257-4712-bc69-f983470d44ca None admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:02.838393 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6665d-3257-4712-bc69-f983470d44ca None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:02.838660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6665d-3257-4712-bc69-f983470d44ca None admin] PATH_INFO: `/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/5c96e9f796814f1e8079f3e16073c2eb/roles/8b837ae2ed57479a93acb638b80e2d67` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:02.845714 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 200/414] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:02.852080 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1dc1f3-b2c4-4bd8-b2a8-c9995a0451c2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:02.857191 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6665d-3257-4712-bc69-f983470d44ca None admin] RBAC: Authorizing `identity:create_grant(project_id=2a5a64cc8aa04972bb464407cb0312b4, user_id=5c96e9f796814f1e8079f3e16073c2eb, role_id=8b837ae2ed57479a93acb638b80e2d67)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:02.858246 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6665d-3257-4712-bc69-f983470d44ca None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:02.892966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 215/415] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:36:02 2022] PUT /identity/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/5c96e9f796814f1e8079f3e16073c2eb/roles/8b837ae2ed57479a93acb638b80e2d67 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:36:02.941631 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:03.022489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1dc1f3-b2c4-4bd8-b2a8-c9995a0451c2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:03.092344 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961444e-7e96-4e02-ab4c-626706a8f02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:03.093356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961444e-7e96-4e02-ab4c-626706a8f02f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:03.093856 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961444e-7e96-4e02-ab4c-626706a8f02f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:03.094320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961444e-7e96-4e02-ab4c-626706a8f02f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:03.109005 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961444e-7e96-4e02-ab4c-626706a8f02f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:03.112573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961444e-7e96-4e02-ab4c-626706a8f02f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-3961444e-7e96-4e02-ab4c-626706a8f02f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:03.136719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:03.137649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:03.138478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:03.138478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:03.138478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:03.138478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:03.138478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:03.138478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:03.138478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:03.138478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:03.140405 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 201/416] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:36:03.146593 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6665d-3257-4712-bc69-f983470d44ca None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:03.372474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6665d-3257-4712-bc69-f983470d44ca None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:03.388772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:03.459743 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12109c-116f-4545-b96b-62fbe2d8a1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:03.460835 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12109c-116f-4545-b96b-62fbe2d8a1ca None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:03.461323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12109c-116f-4545-b96b-62fbe2d8a1ca None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:03.461794 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12109c-116f-4545-b96b-62fbe2d8a1ca None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:03.463061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12109c-116f-4545-b96b-62fbe2d8a1ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:03.463589 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12109c-116f-4545-b96b-62fbe2d8a1ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:03.466438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12109c-116f-4545-b96b-62fbe2d8a1ca None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:03.497583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 216/417] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:03.504528 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961444e-7e96-4e02-ab4c-626706a8f02f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:03.693768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961444e-7e96-4e02-ab4c-626706a8f02f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:03.737991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:03.754789 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55972f-1f3c-481a-956e-c274719184cd None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:03.755269 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55972f-1f3c-481a-956e-c274719184cd None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:03.755728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55972f-1f3c-481a-956e-c274719184cd None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:03.758542 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 217/418] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:03.765202 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab2996-6d49-4ffe-840a-50ee011893e7 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:03.765851 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab2996-6d49-4ffe-840a-50ee011893e7 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:03.766334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab2996-6d49-4ffe-840a-50ee011893e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:03.786151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-b9ab2996-6d49-4ffe-840a-50ee011893e7 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:03.788218 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b9ab2996-6d49-4ffe-840a-50ee011893e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:03.801270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed03964-19fb-474d-bea6-b2357ee6cc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:03.802358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed03964-19fb-474d-bea6-b2357ee6cc2f None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:03.802798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed03964-19fb-474d-bea6-b2357ee6cc2f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:03.803265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed03964-19fb-474d-bea6-b2357ee6cc2f None admin] PATH_INFO: `/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/90985ec8c43a4d7895942731dc1d002c/roles/8b837ae2ed57479a93acb638b80e2d67` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:03.831566 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed03964-19fb-474d-bea6-b2357ee6cc2f None admin] RBAC: Authorizing `identity:create_grant(project_id=2a5a64cc8aa04972bb464407cb0312b4, user_id=90985ec8c43a4d7895942731dc1d002c, role_id=8b837ae2ed57479a93acb638b80e2d67)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:03.834323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed03964-19fb-474d-bea6-b2357ee6cc2f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:03.859942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 202/419] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:36:03 2022] PUT /identity/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/90985ec8c43a4d7895942731dc1d002c/roles/8b837ae2ed57479a93acb638b80e2d67 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:36:03.970878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 218/420] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 15 13:36:04.138153 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7913da-ec12-48dc-9021-63c13df3986b None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:04.138896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7913da-ec12-48dc-9021-63c13df3986b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:04.139494 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7913da-ec12-48dc-9021-63c13df3986b None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:04.141749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 203/421] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:04.147400 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c2c36-51ae-4a6b-a62e-44007dc323a7 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:04.147922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c2c36-51ae-4a6b-a62e-44007dc323a7 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:04.148371 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c2c36-51ae-4a6b-a62e-44007dc323a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:04.168468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-052c2c36-51ae-4a6b-a62e-44007dc323a7 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:04.169933 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-052c2c36-51ae-4a6b-a62e-44007dc323a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:04.289853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 219/422] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:36:04.304808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7913da-ec12-48dc-9021-63c13df3986b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:04.564345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7913da-ec12-48dc-9021-63c13df3986b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:04.611422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c2c36-51ae-4a6b-a62e-44007dc323a7 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:04.636507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b2f75-8e2a-4c6c-911b-d0610c37e307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:04.637609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b2f75-8e2a-4c6c-911b-d0610c37e307 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:04.638052 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b2f75-8e2a-4c6c-911b-d0610c37e307 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:04.638488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b2f75-8e2a-4c6c-911b-d0610c37e307 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:04.675502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b2f75-8e2a-4c6c-911b-d0610c37e307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-ba4b2f75-8e2a-4c6c-911b-d0610c37e307 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:36:04.690248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:04.691277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:04.692098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:04.692098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:04.692098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:04.692098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:04.692098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:36:04.692098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:36:04.692098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:36:04.692098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:36:04.692098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:36:04.692098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:36:04.692098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:04.692098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:04.698641 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 204/423] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:36:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:04.705138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:04.705944 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b2f75-8e2a-4c6c-911b-d0610c37e307 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:04.764126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:04.895689 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b2f75-8e2a-4c6c-911b-d0610c37e307 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:04.935420 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c2c36-51ae-4a6b-a62e-44007dc323a7 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:04.960284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6003ad-9c1a-41d4-91df-d399a2a5a138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:04.961292 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6003ad-9c1a-41d4-91df-d399a2a5a138 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:04.961766 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6003ad-9c1a-41d4-91df-d399a2a5a138 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:04.962226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6003ad-9c1a-41d4-91df-d399a2a5a138 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:04.963333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6003ad-9c1a-41d4-91df-d399a2a5a138 None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:04.963850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6003ad-9c1a-41d4-91df-d399a2a5a138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:04.966581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6003ad-9c1a-41d4-91df-d399a2a5a138 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:04.983237 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 205/424] 10.222.0.36 () {56 vars in 1103 bytes} [Sat Oct 15 13:36:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:04.997308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6003ad-9c1a-41d4-91df-d399a2a5a138 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:05.001569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c128a-25e1-423f-9537-a1c78761ed10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:05.002511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c128a-25e1-423f-9537-a1c78761ed10 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:05.002791 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c128a-25e1-423f-9537-a1c78761ed10 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:05.003065 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c128a-25e1-423f-9537-a1c78761ed10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:05.020681 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c128a-25e1-423f-9537-a1c78761ed10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-615c128a-25e1-423f-9537-a1c78761ed10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:36:05.025608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:05.026513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:05.027302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:05.027302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:05.027302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:05.027302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:05.027302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:36:05.027302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:36:05.027302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:36:05.027302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:36:05.027302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:36:05.027302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:36:05.027302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:05.027302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:05.027302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 220/425] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:36:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:05.037309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c128a-25e1-423f-9537-a1c78761ed10 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:05.210581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:05.212243 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:05.223183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6003ad-9c1a-41d4-91df-d399a2a5a138 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:05.249268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c128a-25e1-423f-9537-a1c78761ed10 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:05.291278 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c641441-a1a2-4980-82c2-677e4fc17ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:05.291990 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c641441-a1a2-4980-82c2-677e4fc17ca0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:05.292272 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c641441-a1a2-4980-82c2-677e4fc17ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:05.292545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c641441-a1a2-4980-82c2-677e4fc17ca0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:05.321140 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c641441-a1a2-4980-82c2-677e4fc17ca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:05.322648 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c641441-a1a2-4980-82c2-677e4fc17ca0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:05.335023 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e8072-1556-4d2a-9efb-c247daa34df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:05.335660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e8072-1556-4d2a-9efb-c247daa34df3 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:05.335972 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e8072-1556-4d2a-9efb-c247daa34df3 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:05.336245 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e8072-1556-4d2a-9efb-c247daa34df3 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:05.336927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e8072-1556-4d2a-9efb-c247daa34df3 None admin] RBAC: Adding query filter params (name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:05.337227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e8072-1556-4d2a-9efb-c247daa34df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:05.338631 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e8072-1556-4d2a-9efb-c247daa34df3 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:05.356352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 221/426] 10.222.0.36 () {56 vars in 1103 bytes} [Sat Oct 15 13:36:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-7c641441-a1a2-4980-82c2-677e4fc17ca0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:05.356943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:05.357779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:05.358628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:05.358628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:05.358628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:05.358628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 206/427] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:05.365203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e8072-1556-4d2a-9efb-c247daa34df3 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:05.366580 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c641441-a1a2-4980-82c2-677e4fc17ca0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:05.506437 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e8072-1556-4d2a-9efb-c247daa34df3 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:05.562130 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c641441-a1a2-4980-82c2-677e4fc17ca0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:05.584858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b8955-48bd-4f6d-a8c4-f1e6856ba5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:05.585533 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b8955-48bd-4f6d-a8c4-f1e6856ba5a4 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:05.585813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b8955-48bd-4f6d-a8c4-f1e6856ba5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:05.586088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b8955-48bd-4f6d-a8c4-f1e6856ba5a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:05.624841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b8955-48bd-4f6d-a8c4-f1e6856ba5a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:05.626741 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b8955-48bd-4f6d-a8c4-f1e6856ba5a4 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:05.660095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc179a1d-2340-4360-9d96-d901b0ef7706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:05.661144 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc179a1d-2340-4360-9d96-d901b0ef7706 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:05.661607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc179a1d-2340-4360-9d96-d901b0ef7706 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:05.662074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc179a1d-2340-4360-9d96-d901b0ef7706 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:05.663218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc179a1d-2340-4360-9d96-d901b0ef7706 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:05.663715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc179a1d-2340-4360-9d96-d901b0ef7706 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:05.666436 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc179a1d-2340-4360-9d96-d901b0ef7706 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-198b8955-48bd-4f6d-a8c4-f1e6856ba5a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:05.683930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:05.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:05.685694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:05.685694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:05.685694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:05.685694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 222/428] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:36:05.685694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:05.699925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b8955-48bd-4f6d-a8c4-f1e6856ba5a4 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:05.717365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 207/429] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:05.726642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc179a1d-2340-4360-9d96-d901b0ef7706 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:05.752831 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:05.933461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b8955-48bd-4f6d-a8c4-f1e6856ba5a4 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:05.992459 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed5678-4e6b-4765-a6ca-4815ff2445f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:05.993438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed5678-4e6b-4765-a6ca-4815ff2445f2 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:05.993438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed5678-4e6b-4765-a6ca-4815ff2445f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:05.993671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed5678-4e6b-4765-a6ca-4815ff2445f2 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:05.994257 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed5678-4e6b-4765-a6ca-4815ff2445f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:05.994561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed5678-4e6b-4765-a6ca-4815ff2445f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:05.996346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed5678-4e6b-4765-a6ca-4815ff2445f2 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:06.014590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 223/430] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:06.016215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f031013-8bc0-4b15-951c-0fbb42a2f71e None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:06.016543 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f031013-8bc0-4b15-951c-0fbb42a2f71e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:06.016833 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f031013-8bc0-4b15-951c-0fbb42a2f71e None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:06.025222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 224/431] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:06.025222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304693c6-bc53-424c-b842-c531411d4b3e None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:06.025222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304693c6-bc53-424c-b842-c531411d4b3e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:06.025222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304693c6-bc53-424c-b842-c531411d4b3e None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:06.025222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 225/432] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:06.029194 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304693c6-bc53-424c-b842-c531411d4b3e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:06.031834 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc179a1d-2340-4360-9d96-d901b0ef7706 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:06.101195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c45da6-6872-464d-b725-849ee430a6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:06.102122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c45da6-6872-464d-b725-849ee430a6aa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:06.102519 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c45da6-6872-464d-b725-849ee430a6aa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:06.102967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c45da6-6872-464d-b725-849ee430a6aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:06.115458 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c45da6-6872-464d-b725-849ee430a6aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:06.121168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c45da6-6872-464d-b725-849ee430a6aa None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:06.125940 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-88c45da6-6872-464d-b725-849ee430a6aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:06.138491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:06.139453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:06.140374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:06.140374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:06.140374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:06.140374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:06.140374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:06.140374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:06.140374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:06.140374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:06.140374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 13:36:06.140374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:06.143253 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 208/433] 10.222.0.36 () {56 vars in 1093 bytes} [Sat Oct 15 13:36:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:06.144700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b4f4b-a6c9-4805-8d59-f718944c8159 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:06.145010 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b4f4b-a6c9-4805-8d59-f718944c8159 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:06.145277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b4f4b-a6c9-4805-8d59-f718944c8159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:06.179012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c53b4f4b-a6c9-4805-8d59-f718944c8159 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:06.181028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c53b4f4b-a6c9-4805-8d59-f718944c8159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:06.228570 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304693c6-bc53-424c-b842-c531411d4b3e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:06.252933 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:06.279912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 209/434] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:36:06.279912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b60b91-5dc1-4d1b-93e4-dad36f9d361b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:06.279912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b60b91-5dc1-4d1b-93e4-dad36f9d361b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:06.279912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b60b91-5dc1-4d1b-93e4-dad36f9d361b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:06.313113 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-30b60b91-5dc1-4d1b-93e4-dad36f9d361b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:06.315191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-30b60b91-5dc1-4d1b-93e4-dad36f9d361b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:06.317395 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9949e-29b3-49aa-a1ea-8eed3a54104b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:06.318384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9949e-29b3-49aa-a1ea-8eed3a54104b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:06.318857 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9949e-29b3-49aa-a1ea-8eed3a54104b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:06.319292 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9949e-29b3-49aa-a1ea-8eed3a54104b None admin] PATH_INFO: `/v3/users/swift` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:06.332777 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9949e-29b3-49aa-a1ea-8eed3a54104b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:06.335267 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9949e-29b3-49aa-a1ea-8eed3a54104b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-2af9949e-29b3-49aa-a1ea-8eed3a54104b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:06.357273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:06.359598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:06.362398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:06.362398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:06.362398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:06.362398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:06.362398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:06.362398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:06.362398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:06.362398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:06.362398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 13:36:06.362398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:06.362398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 226/435] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:36:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:06.362398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9949e-29b3-49aa-a1ea-8eed3a54104b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:06.464562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 210/436] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:36:06.466808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b60b91-5dc1-4d1b-93e4-dad36f9d361b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:06.480198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9949e-29b3-49aa-a1ea-8eed3a54104b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:06.503849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:06.530360 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbf569-b5d7-4e20-9d75-044cdbe99b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:06.532673 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbf569-b5d7-4e20-9d75-044cdbe99b57 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:06.537034 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbf569-b5d7-4e20-9d75-044cdbe99b57 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:06.537822 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbf569-b5d7-4e20-9d75-044cdbe99b57 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:06.541626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccbf569-b5d7-4e20-9d75-044cdbe99b57 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:06.542573 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccbf569-b5d7-4e20-9d75-044cdbe99b57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:06.545491 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccbf569-b5d7-4e20-9d75-044cdbe99b57 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:06.586323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 227/437] 10.222.0.36 () {56 vars in 1133 bytes} [Sat Oct 15 13:36:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:06.589089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84295646-6b7b-4ba9-9fa6-2d70cb98332b None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:06.589839 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84295646-6b7b-4ba9-9fa6-2d70cb98332b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:06.590570 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84295646-6b7b-4ba9-9fa6-2d70cb98332b None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:06.593639 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 228/438] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:06.595638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84295646-6b7b-4ba9-9fa6-2d70cb98332b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:06.650062 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b60b91-5dc1-4d1b-93e4-dad36f9d361b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:06.680094 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:06.695099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1bdee-fc04-4ecd-a9e6-675f67eaa9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:06.695726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1bdee-fc04-4ecd-a9e6-675f67eaa9a4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:06.696120 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1bdee-fc04-4ecd-a9e6-675f67eaa9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:06.696361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1bdee-fc04-4ecd-a9e6-675f67eaa9a4 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:06.697048 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1bdee-fc04-4ecd-a9e6-675f67eaa9a4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:06.697843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1bdee-fc04-4ecd-a9e6-675f67eaa9a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:06.698966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1bdee-fc04-4ecd-a9e6-675f67eaa9a4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:06.726242 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 211/439] 10.222.0.36 () {56 vars in 1135 bytes} [Sat Oct 15 13:36:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:06.726618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a7aa3-0189-4e85-9a6b-416072e6e9cc None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:06.727079 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a7aa3-0189-4e85-9a6b-416072e6e9cc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:06.727537 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a7aa3-0189-4e85-9a6b-416072e6e9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:06.749387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84295646-6b7b-4ba9-9fa6-2d70cb98332b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:06.750485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-505a7aa3-0189-4e85-9a6b-416072e6e9cc None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:06.752164 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-505a7aa3-0189-4e85-9a6b-416072e6e9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:06.795887 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32911475-8982-44fa-ba21-51b4f6a9fabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:06.796530 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32911475-8982-44fa-ba21-51b4f6a9fabb None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:06.796805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32911475-8982-44fa-ba21-51b4f6a9fabb None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:06.797070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32911475-8982-44fa-ba21-51b4f6a9fabb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:06.840017 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 212/440] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:36:06.841982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a7aa3-0189-4e85-9a6b-416072e6e9cc None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:06.861875 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:06.903345 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32911475-8982-44fa-ba21-51b4f6a9fabb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:06.906299 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32911475-8982-44fa-ba21-51b4f6a9fabb None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:07.025292 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a7aa3-0189-4e85-9a6b-416072e6e9cc None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:07.052007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:07.069451 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 229/441] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:36:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:36:07.070213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32911475-8982-44fa-ba21-51b4f6a9fabb None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:07.097314 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b94a54-6f04-412d-930b-22028c4cb690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:07.098687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b94a54-6f04-412d-930b-22028c4cb690 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:07.099392 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b94a54-6f04-412d-930b-22028c4cb690 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:07.099793 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b94a54-6f04-412d-930b-22028c4cb690 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:07.119078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b94a54-6f04-412d-930b-22028c4cb690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-33b94a54-6f04-412d-930b-22028c4cb690 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:36:07.128123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:07.129105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:07.129927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:07.129927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:07.129927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:07.129927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:07.129927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:36:07.129927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:36:07.129927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:36:07.129927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:36:07.129927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:36:07.129927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:36:07.129927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:07.129927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:07.130551 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 213/442] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:36:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:07.131490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b94a54-6f04-412d-930b-22028c4cb690 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:07.192785 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:07.224326 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32911475-8982-44fa-ba21-51b4f6a9fabb None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:07.278570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b94a54-6f04-412d-930b-22028c4cb690 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:07.289562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5909f1-2af1-40fb-b5e1-8eb8266484cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:07.290607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5909f1-2af1-40fb-b5e1-8eb8266484cf None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:07.290886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5909f1-2af1-40fb-b5e1-8eb8266484cf None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:07.291155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5909f1-2af1-40fb-b5e1-8eb8266484cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:07.305427 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5909f1-2af1-40fb-b5e1-8eb8266484cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:07.308186 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5909f1-2af1-40fb-b5e1-8eb8266484cf None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:07.317756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48b860-0a57-484b-8ead-9363d937f784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:07.318400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48b860-0a57-484b-8ead-9363d937f784 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:07.318671 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48b860-0a57-484b-8ead-9363d937f784 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:07.318936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48b860-0a57-484b-8ead-9363d937f784 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:07.330499 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48b860-0a57-484b-8ead-9363d937f784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-ee5909f1-2af1-40fb-b5e1-8eb8266484cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:07.335056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:07.336189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:07.337040 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:07.337040 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:07.337040 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-3d48b860-0a57-484b-8ead-9363d937f784 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:36:07.337320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:07.340083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:07.342201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:07.342201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:07.342201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:07.342201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:07.342201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:36:07.342201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:36:07.342201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:36:07.342201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:36:07.342201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:36:07.342201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:36:07.342201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:07.342201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:07.342201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 214/444] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:36:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:07.343741 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 230/444] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:07.343982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48b860-0a57-484b-8ead-9363d937f784 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:07.345060 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5909f1-2af1-40fb-b5e1-8eb8266484cf None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:07.396880 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:07.475711 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5909f1-2af1-40fb-b5e1-8eb8266484cf None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:07.510747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48b860-0a57-484b-8ead-9363d937f784 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:07.514908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:07.521168 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635184f-4a17-4121-bf24-b5c11b22e10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:07.521906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635184f-4a17-4121-bf24-b5c11b22e10e None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:07.522277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635184f-4a17-4121-bf24-b5c11b22e10e None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:07.522627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635184f-4a17-4121-bf24-b5c11b22e10e None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:07.523421 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635184f-4a17-4121-bf24-b5c11b22e10e None admin] RBAC: Adding query filter params (name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:07.523803 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635184f-4a17-4121-bf24-b5c11b22e10e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:07.525714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635184f-4a17-4121-bf24-b5c11b22e10e None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:07.535553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 231/445] 10.222.0.36 () {56 vars in 1103 bytes} [Sat Oct 15 13:36:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:07.536425 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635184f-4a17-4121-bf24-b5c11b22e10e None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:07.559210 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab3ce9-f4e3-4170-a5b1-ef46840c8269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:07.559982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab3ce9-f4e3-4170-a5b1-ef46840c8269 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:07.560326 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab3ce9-f4e3-4170-a5b1-ef46840c8269 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:07.560656 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab3ce9-f4e3-4170-a5b1-ef46840c8269 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:07.573702 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab3ce9-f4e3-4170-a5b1-ef46840c8269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:07.575224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab3ce9-f4e3-4170-a5b1-ef46840c8269 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-34ab3ce9-f4e3-4170-a5b1-ef46840c8269 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:07.590431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:07.592300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:07.594131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:07.594131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:07.594131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:07.594131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 215/446] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:07.594131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab3ce9-f4e3-4170-a5b1-ef46840c8269 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:07.681204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635184f-4a17-4121-bf24-b5c11b22e10e None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:07.712277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:07.727752 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270a738-481e-45ae-9b74-68817e17e129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:07.728556 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270a738-481e-45ae-9b74-68817e17e129 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:07.728890 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270a738-481e-45ae-9b74-68817e17e129 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:07.729272 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270a738-481e-45ae-9b74-68817e17e129 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:07.731141 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab3ce9-f4e3-4170-a5b1-ef46840c8269 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:07.742224 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270a738-481e-45ae-9b74-68817e17e129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-3270a738-481e-45ae-9b74-68817e17e129 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:36:07.747730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:07.749841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:07.752012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:07.752012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:07.752012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:07.752012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:07.752012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:36:07.752012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:36:07.752012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:36:07.752012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:36:07.752012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:36:07.752012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:36:07.752012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:07.752012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:07.752012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 232/447] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:36:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:07.752012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270a738-481e-45ae-9b74-68817e17e129 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:07.780577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefdbe69-9331-4adb-a6e5-1a5cedefd518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:07.781320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdbe69-9331-4adb-a6e5-1a5cedefd518 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:07.781659 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdbe69-9331-4adb-a6e5-1a5cedefd518 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:07.781985 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdbe69-9331-4adb-a6e5-1a5cedefd518 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:07.782761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefdbe69-9331-4adb-a6e5-1a5cedefd518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:07.783136 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefdbe69-9331-4adb-a6e5-1a5cedefd518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:07.784823 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefdbe69-9331-4adb-a6e5-1a5cedefd518 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:07.803758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 216/448] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:07.805056 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefdbe69-9331-4adb-a6e5-1a5cedefd518 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:07.836287 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:07.900398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270a738-481e-45ae-9b74-68817e17e129 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:07.939334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14763779-60dd-4be8-b670-527a601e09e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:07.940149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14763779-60dd-4be8-b670-527a601e09e4 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:07.940431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14763779-60dd-4be8-b670-527a601e09e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:07.940704 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14763779-60dd-4be8-b670-527a601e09e4 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:07.941389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14763779-60dd-4be8-b670-527a601e09e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:07.941697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14763779-60dd-4be8-b670-527a601e09e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:07.942125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefdbe69-9331-4adb-a6e5-1a5cedefd518 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:07.943114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14763779-60dd-4be8-b670-527a601e09e4 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:07.952129 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 233/449] 10.222.0.36 () {56 vars in 1103 bytes} [Sat Oct 15 13:36:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:07.953069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14763779-60dd-4be8-b670-527a601e09e4 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:07.996191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fef63-6e63-4f0c-884e-ade798ac7fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:07.996850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fef63-6e63-4f0c-884e-ade798ac7fc9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:07.997126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fef63-6e63-4f0c-884e-ade798ac7fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:07.997391 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fef63-6e63-4f0c-884e-ade798ac7fc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:08.009207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324fef63-6e63-4f0c-884e-ade798ac7fc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:08.010736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324fef63-6e63-4f0c-884e-ade798ac7fc9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:08.013903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-324fef63-6e63-4f0c-884e-ade798ac7fc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:08.028395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:08.029895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:08.030783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:08.030783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:08.030783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:08.038199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 217/450] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:08.038199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fef63-6e63-4f0c-884e-ade798ac7fc9 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:08.119559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14763779-60dd-4be8-b670-527a601e09e4 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:08.137083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:08.159846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fef63-6e63-4f0c-884e-ade798ac7fc9 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:08.171858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67b52d-5ce4-43af-a554-58372044610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:08.172753 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67b52d-5ce4-43af-a554-58372044610d None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:08.173347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67b52d-5ce4-43af-a554-58372044610d None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:08.173673 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67b52d-5ce4-43af-a554-58372044610d None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:08.174534 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67b52d-5ce4-43af-a554-58372044610d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:08.175004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67b52d-5ce4-43af-a554-58372044610d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:08.176779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67b52d-5ce4-43af-a554-58372044610d None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:08.192144 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 234/451] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:08.193598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67b52d-5ce4-43af-a554-58372044610d None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:08.208972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e48a6d-986d-4a80-ad06-bbd940c26f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:08.209976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e48a6d-986d-4a80-ad06-bbd940c26f7a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:08.210432 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e48a6d-986d-4a80-ad06-bbd940c26f7a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:08.210878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e48a6d-986d-4a80-ad06-bbd940c26f7a None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:08.211988 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e48a6d-986d-4a80-ad06-bbd940c26f7a None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:08.212514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e48a6d-986d-4a80-ad06-bbd940c26f7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:08.214915 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e48a6d-986d-4a80-ad06-bbd940c26f7a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:08.229830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 218/452] 10.222.0.36 () {56 vars in 1103 bytes} [Sat Oct 15 13:36:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:08.231469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e48a6d-986d-4a80-ad06-bbd940c26f7a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:08.307245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:08.316118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67b52d-5ce4-43af-a554-58372044610d None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:08.358628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffde3ec-1853-4fc6-a2ac-d529b0ff8dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:08.359474 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e48a6d-986d-4a80-ad06-bbd940c26f7a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:08.359664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffde3ec-1853-4fc6-a2ac-d529b0ff8dca None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:08.359867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffde3ec-1853-4fc6-a2ac-d529b0ff8dca None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:08.360169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffde3ec-1853-4fc6-a2ac-d529b0ff8dca None admin] PATH_INFO: `/v3/projects/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:08.369107 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffde3ec-1853-4fc6-a2ac-d529b0ff8dca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:08.370838 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffde3ec-1853-4fc6-a2ac-d529b0ff8dca None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-bffde3ec-1853-4fc6-a2ac-d529b0ff8dca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:08.385501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:08.387409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:08.389354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:08.389354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:08.389354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:08.389354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:08.389354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:08.389354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:08.389354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:08.389354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:08.389354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 235/453] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:08.389354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffde3ec-1853-4fc6-a2ac-d529b0ff8dca None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:08.411582 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc695b-5106-488a-9c49-eb833d5f21aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:08.412402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc695b-5106-488a-9c49-eb833d5f21aa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:08.412778 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc695b-5106-488a-9c49-eb833d5f21aa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:08.413151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc695b-5106-488a-9c49-eb833d5f21aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:08.426017 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc695b-5106-488a-9c49-eb833d5f21aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:08.427517 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc695b-5106-488a-9c49-eb833d5f21aa None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:08.431452 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-bccc695b-5106-488a-9c49-eb833d5f21aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:08.443847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:08.445735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:08.447307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:08.447307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:08.447307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:08.447307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 219/454] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:08.447307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc695b-5106-488a-9c49-eb833d5f21aa None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:08.541305 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffde3ec-1853-4fc6-a2ac-d529b0ff8dca None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:08.599896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc695b-5106-488a-9c49-eb833d5f21aa None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:08.604290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072dc04-a110-4544-b72a-f50e65106594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:08.605061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072dc04-a110-4544-b72a-f50e65106594 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:08.605349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072dc04-a110-4544-b72a-f50e65106594 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:08.605624 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072dc04-a110-4544-b72a-f50e65106594 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:08.606306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072dc04-a110-4544-b72a-f50e65106594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:08.607611 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072dc04-a110-4544-b72a-f50e65106594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:08.608093 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072dc04-a110-4544-b72a-f50e65106594 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:08.620573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:08.623750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 236/455] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:08.624875 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072dc04-a110-4544-b72a-f50e65106594 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:08.656478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb8f8b-b17d-4c49-bbfa-7e0eaac11983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:08.658287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8f8b-b17d-4c49-bbfa-7e0eaac11983 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:08.659007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8f8b-b17d-4c49-bbfa-7e0eaac11983 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:08.659502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8f8b-b17d-4c49-bbfa-7e0eaac11983 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:08.675601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb8f8b-b17d-4c49-bbfa-7e0eaac11983 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:08.677440 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb8f8b-b17d-4c49-bbfa-7e0eaac11983 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-fdbb8f8b-b17d-4c49-bbfa-7e0eaac11983 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:08.692951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:08.694532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:08.696002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:08.696002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:08.696002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:08.696002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:08.696002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:08.696002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:08.696002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:08.696002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:08.696002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 220/456] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:08.696002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb8f8b-b17d-4c49-bbfa-7e0eaac11983 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:08.774334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:08.785842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072dc04-a110-4544-b72a-f50e65106594 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:08.838507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb8f8b-b17d-4c49-bbfa-7e0eaac11983 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:08.851496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31cc2d-e510-4254-bb8a-b84bd76dec24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:08.852307 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31cc2d-e510-4254-bb8a-b84bd76dec24 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:08.852655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31cc2d-e510-4254-bb8a-b84bd76dec24 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:08.852997 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31cc2d-e510-4254-bb8a-b84bd76dec24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:08.868388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31cc2d-e510-4254-bb8a-b84bd76dec24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:08.869939 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31cc2d-e510-4254-bb8a-b84bd76dec24 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:08.895669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b3637-e60d-4322-85d6-0051e6d2972b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:08.896677 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b3637-e60d-4322-85d6-0051e6d2972b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:08.897121 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b3637-e60d-4322-85d6-0051e6d2972b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:08.897558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b3637-e60d-4322-85d6-0051e6d2972b None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:08.898711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b3637-e60d-4322-85d6-0051e6d2972b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:08.899181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b3637-e60d-4322-85d6-0051e6d2972b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-2a31cc2d-e510-4254-bb8a-b84bd76dec24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:08.900994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:08.901859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:08.904612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:08.904612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:08.904612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:08.904853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b3637-e60d-4322-85d6-0051e6d2972b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:08.904975 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 237/457] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:08.904975 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31cc2d-e510-4254-bb8a-b84bd76dec24 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:08.927388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 221/458] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:08.928505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b3637-e60d-4322-85d6-0051e6d2972b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:08.971524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:09.060363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b3637-e60d-4322-85d6-0051e6d2972b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:09.093365 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31cc2d-e510-4254-bb8a-b84bd76dec24 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:09.102605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d78a55-870b-47c0-b351-2739322cfd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:09.103266 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d78a55-870b-47c0-b351-2739322cfd86 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:09.103539 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d78a55-870b-47c0-b351-2739322cfd86 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:09.103805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d78a55-870b-47c0-b351-2739322cfd86 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:09.114264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d78a55-870b-47c0-b351-2739322cfd86 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:09.117035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d78a55-870b-47c0-b351-2739322cfd86 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-d2d78a55-870b-47c0-b351-2739322cfd86 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:09.132342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:09.133966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:09.134860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:09.134860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:09.134860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:09.134860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:09.134860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:09.134860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:09.134860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:09.134860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:09.134860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 13:36:09.134860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:09.134860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 222/459] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:36:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:09.134860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d78a55-870b-47c0-b351-2739322cfd86 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:09.149900 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b6e7a-3c46-4003-a94e-2a2f35ea1750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:09.150719 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b6e7a-3c46-4003-a94e-2a2f35ea1750 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:09.150885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b6e7a-3c46-4003-a94e-2a2f35ea1750 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:09.151121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b6e7a-3c46-4003-a94e-2a2f35ea1750 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:09.151788 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b6e7a-3c46-4003-a94e-2a2f35ea1750 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:09.152140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b6e7a-3c46-4003-a94e-2a2f35ea1750 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:09.153558 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b6e7a-3c46-4003-a94e-2a2f35ea1750 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:09.174184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 238/460] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:09.175258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b6e7a-3c46-4003-a94e-2a2f35ea1750 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:09.190931 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:09.260650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:09.279281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d78a55-870b-47c0-b351-2739322cfd86 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:09.310334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b6e7a-3c46-4003-a94e-2a2f35ea1750 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:09.330742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c97e5-b6c5-42d7-9ade-938db7339aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:09.331604 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c97e5-b6c5-42d7-9ade-938db7339aaa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:09.331604 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c97e5-b6c5-42d7-9ade-938db7339aaa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:09.331919 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c97e5-b6c5-42d7-9ade-938db7339aaa None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:09.332593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c97e5-b6c5-42d7-9ade-938db7339aaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:09.332886 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c97e5-b6c5-42d7-9ade-938db7339aaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:09.335308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c97e5-b6c5-42d7-9ade-938db7339aaa None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:09.359749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 223/461] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:09.359749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c97e5-b6c5-42d7-9ade-938db7339aaa None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:09.360139 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08e9c3-758b-4ba6-94b8-4f39ed99d940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:09.360784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08e9c3-758b-4ba6-94b8-4f39ed99d940 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:09.361072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08e9c3-758b-4ba6-94b8-4f39ed99d940 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:09.361344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08e9c3-758b-4ba6-94b8-4f39ed99d940 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:09.362016 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08e9c3-758b-4ba6-94b8-4f39ed99d940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:09.362331 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08e9c3-758b-4ba6-94b8-4f39ed99d940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:09.364014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08e9c3-758b-4ba6-94b8-4f39ed99d940 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:09.377650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 239/462] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:09.379416 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08e9c3-758b-4ba6-94b8-4f39ed99d940 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:09.458073 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:09.506833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c97e5-b6c5-42d7-9ade-938db7339aaa None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:09.520136 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08e9c3-758b-4ba6-94b8-4f39ed99d940 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:09.544676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c282c5-ecdd-495d-af2f-77c1fa92bdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:09.545416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c282c5-ecdd-495d-af2f-77c1fa92bdeb None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:09.545746 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c282c5-ecdd-495d-af2f-77c1fa92bdeb None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:09.546085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c282c5-ecdd-495d-af2f-77c1fa92bdeb None admin] PATH_INFO: `/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/d011a40951564323bf7a9c5601df1470/roles/8b837ae2ed57479a93acb638b80e2d67` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:09.562447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04cc6b-89e3-4b95-b847-0d3b82754269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:09.563068 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04cc6b-89e3-4b95-b847-0d3b82754269 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:09.563348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04cc6b-89e3-4b95-b847-0d3b82754269 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:09.563913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04cc6b-89e3-4b95-b847-0d3b82754269 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:09.564299 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c282c5-ecdd-495d-af2f-77c1fa92bdeb None admin] RBAC: Authorizing `identity:create_grant(project_id=2a5a64cc8aa04972bb464407cb0312b4, user_id=d011a40951564323bf7a9c5601df1470, role_id=8b837ae2ed57479a93acb638b80e2d67)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:09.565095 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04cc6b-89e3-4b95-b847-0d3b82754269 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:09.565411 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04cc6b-89e3-4b95-b847-0d3b82754269 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:09.566002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c282c5-ecdd-495d-af2f-77c1fa92bdeb None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:09.566878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04cc6b-89e3-4b95-b847-0d3b82754269 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:09.578750 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:09.594093 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 224/463] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:36:09 2022] PUT /identity/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/d011a40951564323bf7a9c5601df1470/roles/8b837ae2ed57479a93acb638b80e2d67 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:36:09.594093 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c282c5-ecdd-495d-af2f-77c1fa92bdeb None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:09.610130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 240/464] 10.222.0.36 () {56 vars in 1137 bytes} [Sat Oct 15 13:36:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:09.610130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04cc6b-89e3-4b95-b847-0d3b82754269 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:09.761951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c282c5-ecdd-495d-af2f-77c1fa92bdeb None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:09.781171 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04cc6b-89e3-4b95-b847-0d3b82754269 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:09.823681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf35330-0ea1-4673-86aa-bac4308eb1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:09.824402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35330-0ea1-4673-86aa-bac4308eb1b7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:09.824700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35330-0ea1-4673-86aa-bac4308eb1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:09.824941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35330-0ea1-4673-86aa-bac4308eb1b7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:09.830784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:09.838973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf35330-0ea1-4673-86aa-bac4308eb1b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:09.841711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf35330-0ea1-4673-86aa-bac4308eb1b7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:09.848970 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed482ee-32f4-4a10-8f26-98413f144592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:09.849904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed482ee-32f4-4a10-8f26-98413f144592 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:09.850353 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed482ee-32f4-4a10-8f26-98413f144592 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:09.850795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed482ee-32f4-4a10-8f26-98413f144592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:09.851711 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed482ee-32f4-4a10-8f26-98413f144592 None admin] RBAC: Adding query filter params (role.id=8b837ae2ed57479a93acb638b80e2d67, scope.project.id=2a5a64cc8aa04972bb464407cb0312b4, user.id=2e13692c1e83413a909d99bb7db2e4b8) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:09.852218 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed482ee-32f4-4a10-8f26-98413f144592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:09.853888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed482ee-32f4-4a10-8f26-98413f144592 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:09.863945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 241/465] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:36:09 2022] GET /identity/v3/role_assignments?user.id=2e13692c1e83413a909d99bb7db2e4b8&scope.project.id=2a5a64cc8aa04972bb464407cb0312b4&role.id=8b837ae2ed57479a93acb638b80e2d67 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-2cf35330-0ea1-4673-86aa-bac4308eb1b7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:09.865191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:09.866787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:09.868578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:09.868578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:09.868578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:09.868578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:09.868578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:09.868578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:09.868578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:09.868578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:09.868578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 15 13:36:09.868578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:09.869571 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed482ee-32f4-4a10-8f26-98413f144592 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:09.879475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 225/466] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:36:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:09.881520 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf35330-0ea1-4673-86aa-bac4308eb1b7 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:10.004525 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:10.105127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed482ee-32f4-4a10-8f26-98413f144592 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:10.113418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf35330-0ea1-4673-86aa-bac4308eb1b7 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:10.168391 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d19088-1d91-4b3f-9498-a49ee9266a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:10.169373 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d19088-1d91-4b3f-9498-a49ee9266a41 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:10.169614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d19088-1d91-4b3f-9498-a49ee9266a41 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:10.169871 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d19088-1d91-4b3f-9498-a49ee9266a41 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:10.182844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69a12a-dbee-412a-b87a-a8ec162df771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:10.183543 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d19088-1d91-4b3f-9498-a49ee9266a41 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:10.184008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69a12a-dbee-412a-b87a-a8ec162df771 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:10.184500 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69a12a-dbee-412a-b87a-a8ec162df771 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:10.184968 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69a12a-dbee-412a-b87a-a8ec162df771 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:10.185419 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d19088-1d91-4b3f-9498-a49ee9266a41 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-66d19088-1d91-4b3f-9498-a49ee9266a41 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:10.200694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:10.202235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:10.203748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:10.203748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:10.203748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:10.203748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:10.203748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:10.203748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:10.203748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:10.203748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:10.203748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 15 13:36:10.203748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:10.203748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 242/467] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:36:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:10.205878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d19088-1d91-4b3f-9498-a49ee9266a41 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:10.286007 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:10.311477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69a12a-dbee-412a-b87a-a8ec162df771 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:10.313366 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69a12a-dbee-412a-b87a-a8ec162df771 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:10.387388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d19088-1d91-4b3f-9498-a49ee9266a41 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:10.395266 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:10.451056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef1e07-3d9b-4577-8404-6859585ff1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:10.451685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef1e07-3d9b-4577-8404-6859585ff1c5 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:10.451996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef1e07-3d9b-4577-8404-6859585ff1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:10.452271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef1e07-3d9b-4577-8404-6859585ff1c5 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:10.453023 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef1e07-3d9b-4577-8404-6859585ff1c5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:10.453325 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef1e07-3d9b-4577-8404-6859585ff1c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:10.455244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef1e07-3d9b-4577-8404-6859585ff1c5 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:10.467220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 226/468] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:36:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:36:10.467220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69a12a-dbee-412a-b87a-a8ec162df771 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:10.484538 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 243/469] 10.222.0.36 () {56 vars in 1137 bytes} [Sat Oct 15 13:36:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:10.485470 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef1e07-3d9b-4577-8404-6859585ff1c5 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:10.585124 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69a12a-dbee-412a-b87a-a8ec162df771 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:10.595664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef1e07-3d9b-4577-8404-6859585ff1c5 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:10.604043 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:10.625075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d466d8b-c53d-4397-8635-65fc45fcb491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:10.626236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d466d8b-c53d-4397-8635-65fc45fcb491 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:10.627343 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d466d8b-c53d-4397-8635-65fc45fcb491 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:10.628088 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d466d8b-c53d-4397-8635-65fc45fcb491 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:10.629160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d466d8b-c53d-4397-8635-65fc45fcb491 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:10.629635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d466d8b-c53d-4397-8635-65fc45fcb491 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:10.631861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d466d8b-c53d-4397-8635-65fc45fcb491 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:10.634857 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bb6ba-0540-4ccb-97be-8669ebdc5994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:10.635908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bb6ba-0540-4ccb-97be-8669ebdc5994 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:10.635908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bb6ba-0540-4ccb-97be-8669ebdc5994 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:10.636104 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bb6ba-0540-4ccb-97be-8669ebdc5994 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:10.647376 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883bb6ba-0540-4ccb-97be-8669ebdc5994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:10.648813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883bb6ba-0540-4ccb-97be-8669ebdc5994 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:10.655719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 227/470] 10.222.0.36 () {56 vars in 1139 bytes} [Sat Oct 15 13:36:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:10.657427 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d466d8b-c53d-4397-8635-65fc45fcb491 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-883bb6ba-0540-4ccb-97be-8669ebdc5994 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:10.665643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:10.666710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:10.667616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:10.667616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:10.667616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:10.671598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 244/471] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:10.672699 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bb6ba-0540-4ccb-97be-8669ebdc5994 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:10.683658 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:10.783803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d466d8b-c53d-4397-8635-65fc45fcb491 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:10.808109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bb6ba-0540-4ccb-97be-8669ebdc5994 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:10.824011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e5a15-5fe7-4bb7-9b0d-6639449ac3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:10.824858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e5a15-5fe7-4bb7-9b0d-6639449ac3c0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:10.825102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e5a15-5fe7-4bb7-9b0d-6639449ac3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:10.825378 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e5a15-5fe7-4bb7-9b0d-6639449ac3c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:10.855913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d353a-22f1-4f59-babe-29a39989082d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:10.857042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d353a-22f1-4f59-babe-29a39989082d None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:10.857912 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d353a-22f1-4f59-babe-29a39989082d None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:10.858348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d353a-22f1-4f59-babe-29a39989082d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:10.905190 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:10.919222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e5a15-5fe7-4bb7-9b0d-6639449ac3c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:10.920890 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e5a15-5fe7-4bb7-9b0d-6639449ac3c0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:10.943459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:10.960429 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d353a-22f1-4f59-babe-29a39989082d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:10.961917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d353a-22f1-4f59-babe-29a39989082d None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:11.056291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 228/472] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:36:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:36:11.057300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e5a15-5fe7-4bb7-9b0d-6639449ac3c0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:11.117796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 245/473] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:36:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:36:11.118695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d353a-22f1-4f59-babe-29a39989082d None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:11.193397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:11.240547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e5a15-5fe7-4bb7-9b0d-6639449ac3c0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:11.266099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:11.273452 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d353a-22f1-4f59-babe-29a39989082d None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:11.289123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27858470-11ea-4b32-9707-d92b92fd5b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:11.289896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27858470-11ea-4b32-9707-d92b92fd5b1d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:11.290269 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27858470-11ea-4b32-9707-d92b92fd5b1d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:11.290634 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27858470-11ea-4b32-9707-d92b92fd5b1d None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:11.291470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27858470-11ea-4b32-9707-d92b92fd5b1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:11.291866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27858470-11ea-4b32-9707-d92b92fd5b1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:11.293434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27858470-11ea-4b32-9707-d92b92fd5b1d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:11.312183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 229/474] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:11.314090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27858470-11ea-4b32-9707-d92b92fd5b1d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:11.338313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c17b90-a2a4-44e9-83e7-72e4bf36d5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:11.339133 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c17b90-a2a4-44e9-83e7-72e4bf36d5a0 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:11.339516 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c17b90-a2a4-44e9-83e7-72e4bf36d5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:11.339914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c17b90-a2a4-44e9-83e7-72e4bf36d5a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:11.352969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c17b90-a2a4-44e9-83e7-72e4bf36d5a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:11.354138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c17b90-a2a4-44e9-83e7-72e4bf36d5a0 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-c9c17b90-a2a4-44e9-83e7-72e4bf36d5a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:11.371019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:11.372837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:11.374178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:11.374178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:11.374178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:11.374178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 246/475] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:11.374178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c17b90-a2a4-44e9-83e7-72e4bf36d5a0 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:11.470105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27858470-11ea-4b32-9707-d92b92fd5b1d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:11.509542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f22882-ff00-4176-9053-14998c86c4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:11.510275 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f22882-ff00-4176-9053-14998c86c4e0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:11.510470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f22882-ff00-4176-9053-14998c86c4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:11.510734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f22882-ff00-4176-9053-14998c86c4e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:11.523369 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f22882-ff00-4176-9053-14998c86c4e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:11.524949 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f22882-ff00-4176-9053-14998c86c4e0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:11.525333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c17b90-a2a4-44e9-83e7-72e4bf36d5a0 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:11.538621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-64f22882-ff00-4176-9053-14998c86c4e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:11.541207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:11.542581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:11.543431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:11.543431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:11.543431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:11.543431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 230/476] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:11.544099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f22882-ff00-4176-9053-14998c86c4e0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:11.572398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3407c-8071-4452-a40e-5907932dc579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:11.573503 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3407c-8071-4452-a40e-5907932dc579 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:11.573991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3407c-8071-4452-a40e-5907932dc579 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:11.574501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3407c-8071-4452-a40e-5907932dc579 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:11.578844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:11.584662 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3407c-8071-4452-a40e-5907932dc579 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:11.587698 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3407c-8071-4452-a40e-5907932dc579 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-51b3407c-8071-4452-a40e-5907932dc579 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:11.602170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:11.603100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:11.604531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:11.604531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:11.604531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:11.604531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:11.604531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:11.604531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:11.604531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:11.604531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:11.606919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 247/477] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:11.608316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3407c-8071-4452-a40e-5907932dc579 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:11.661784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f22882-ff00-4176-9053-14998c86c4e0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:11.698866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be3dc2-ce88-4dab-abe3-ae6ad3aeb6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:11.699489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be3dc2-ce88-4dab-abe3-ae6ad3aeb6db None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:11.699760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be3dc2-ce88-4dab-abe3-ae6ad3aeb6db None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:11.700669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be3dc2-ce88-4dab-abe3-ae6ad3aeb6db None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:11.700800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be3dc2-ce88-4dab-abe3-ae6ad3aeb6db None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:11.700980 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be3dc2-ce88-4dab-abe3-ae6ad3aeb6db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:11.702384 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be3dc2-ce88-4dab-abe3-ae6ad3aeb6db None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:11.715589 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 231/478] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:11.716541 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be3dc2-ce88-4dab-abe3-ae6ad3aeb6db None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:11.730993 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3407c-8071-4452-a40e-5907932dc579 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:11.771456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bae8f0-73eb-432b-80a2-888cc455f648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:11.772485 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bae8f0-73eb-432b-80a2-888cc455f648 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:11.773043 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bae8f0-73eb-432b-80a2-888cc455f648 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:11.773639 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bae8f0-73eb-432b-80a2-888cc455f648 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:11.774524 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bae8f0-73eb-432b-80a2-888cc455f648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:11.775029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bae8f0-73eb-432b-80a2-888cc455f648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:11.776766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bae8f0-73eb-432b-80a2-888cc455f648 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:11.791444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 248/479] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:11.792726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bae8f0-73eb-432b-80a2-888cc455f648 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:11.820543 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:11.832032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be3dc2-ce88-4dab-abe3-ae6ad3aeb6db None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:11.833051 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:11.875776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d93178-7396-4399-8f50-a3056ff4481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:11.876797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d93178-7396-4399-8f50-a3056ff4481f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:11.876911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d93178-7396-4399-8f50-a3056ff4481f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:11.877262 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d93178-7396-4399-8f50-a3056ff4481f None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:11.877994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d93178-7396-4399-8f50-a3056ff4481f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:11.878776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d93178-7396-4399-8f50-a3056ff4481f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:11.879992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d93178-7396-4399-8f50-a3056ff4481f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:11.894562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 232/480] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:11.895455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d93178-7396-4399-8f50-a3056ff4481f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:11.928734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bae8f0-73eb-432b-80a2-888cc455f648 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:11.969588 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa728fee-cf4a-45d5-b8ca-59c037b90327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:11.970853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa728fee-cf4a-45d5-b8ca-59c037b90327 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:11.971389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa728fee-cf4a-45d5-b8ca-59c037b90327 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:11.971862 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa728fee-cf4a-45d5-b8ca-59c037b90327 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:11.980397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa728fee-cf4a-45d5-b8ca-59c037b90327 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:11.982482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa728fee-cf4a-45d5-b8ca-59c037b90327 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-aa728fee-cf4a-45d5-b8ca-59c037b90327 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:11.996314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:11.997231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:11.998059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:11.998059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:11.998059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:11.998059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:11.998059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:11.998059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:11.998059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:11.998059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:12.000674 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 249/481] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:12.001966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa728fee-cf4a-45d5-b8ca-59c037b90327 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:12.099844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d93178-7396-4399-8f50-a3056ff4481f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:12.105515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:12.117543 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa728fee-cf4a-45d5-b8ca-59c037b90327 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:12.156008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf9c8d-9035-4014-9ce4-ac0beaa8d054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:12.157011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf9c8d-9035-4014-9ce4-ac0beaa8d054 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:12.157584 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf9c8d-9035-4014-9ce4-ac0beaa8d054 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:12.158080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf9c8d-9035-4014-9ce4-ac0beaa8d054 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:12.159148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf9c8d-9035-4014-9ce4-ac0beaa8d054 None admin] RBAC: Adding query filter params (role.id=8b837ae2ed57479a93acb638b80e2d67, scope.project.id=2a5a64cc8aa04972bb464407cb0312b4, user.id=5c96e9f796814f1e8079f3e16073c2eb) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:12.159698 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf9c8d-9035-4014-9ce4-ac0beaa8d054 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:12.162641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf9c8d-9035-4014-9ce4-ac0beaa8d054 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:12.165773 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461038cd-46b2-4148-bcb4-993ccc9cf7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:12.167003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461038cd-46b2-4148-bcb4-993ccc9cf7cf None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:12.167493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461038cd-46b2-4148-bcb4-993ccc9cf7cf None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:12.168019 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461038cd-46b2-4148-bcb4-993ccc9cf7cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:12.174977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 250/482] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:36:12 2022] GET /identity/v3/role_assignments?user.id=5c96e9f796814f1e8079f3e16073c2eb&scope.project.id=2a5a64cc8aa04972bb464407cb0312b4&role.id=8b837ae2ed57479a93acb638b80e2d67 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:12.176404 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461038cd-46b2-4148-bcb4-993ccc9cf7cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:12.176522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf9c8d-9035-4014-9ce4-ac0beaa8d054 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:12.178254 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461038cd-46b2-4148-bcb4-993ccc9cf7cf None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-461038cd-46b2-4148-bcb4-993ccc9cf7cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:12.192065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:12.192992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:12.193829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:12.193829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:12.193829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:12.193829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:12.193829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:12.193829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:12.193829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:12.193829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:12.196451 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 233/483] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:12.202988 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461038cd-46b2-4148-bcb4-993ccc9cf7cf None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:12.230197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:12.456177 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf9c8d-9035-4014-9ce4-ac0beaa8d054 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:12.541808 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847df9d-92a1-46ed-959d-fe37354009f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:12.542850 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847df9d-92a1-46ed-959d-fe37354009f8 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:12.543321 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847df9d-92a1-46ed-959d-fe37354009f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:12.543776 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847df9d-92a1-46ed-959d-fe37354009f8 None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:12.545088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6847df9d-92a1-46ed-959d-fe37354009f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:12.545621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6847df9d-92a1-46ed-959d-fe37354009f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:12.548481 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6847df9d-92a1-46ed-959d-fe37354009f8 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:12.575544 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461038cd-46b2-4148-bcb4-993ccc9cf7cf None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:12.586616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 251/484] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:12.598378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847df9d-92a1-46ed-959d-fe37354009f8 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:12.632321 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:12.661373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af4878-a18c-4972-a96c-d3aea968cc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:12.662367 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4878-a18c-4972-a96c-d3aea968cc45 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:12.662881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4878-a18c-4972-a96c-d3aea968cc45 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:12.663340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4878-a18c-4972-a96c-d3aea968cc45 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:12.664309 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af4878-a18c-4972-a96c-d3aea968cc45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:12.664972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af4878-a18c-4972-a96c-d3aea968cc45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:12.666768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af4878-a18c-4972-a96c-d3aea968cc45 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:12.685779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 234/485] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:12.688312 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ff31b-0ac4-4793-af9d-540f33592644 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:12.688878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ff31b-0ac4-4793-af9d-540f33592644 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:12.689355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ff31b-0ac4-4793-af9d-540f33592644 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:12.692307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 235/486] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:12.693762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ff31b-0ac4-4793-af9d-540f33592644 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:12.766811 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847df9d-92a1-46ed-959d-fe37354009f8 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:12.795847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:12.805411 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca07413-bd6b-422d-8769-b9b345ecf41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:12.805915 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca07413-bd6b-422d-8769-b9b345ecf41c None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:12.806162 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca07413-bd6b-422d-8769-b9b345ecf41c None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:12.806354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca07413-bd6b-422d-8769-b9b345ecf41c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:12.807157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca07413-bd6b-422d-8769-b9b345ecf41c None admin] RBAC: Adding query filter params (role.id=8b837ae2ed57479a93acb638b80e2d67, scope.project.id=2a5a64cc8aa04972bb464407cb0312b4, user.id=4f5aabcb1414413a8dce7b76ce701582) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:12.807469 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca07413-bd6b-422d-8769-b9b345ecf41c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:12.809011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca07413-bd6b-422d-8769-b9b345ecf41c None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:12.816127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 252/487] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:36:12 2022] GET /identity/v3/role_assignments?user.id=4f5aabcb1414413a8dce7b76ce701582&scope.project.id=2a5a64cc8aa04972bb464407cb0312b4&role.id=8b837ae2ed57479a93acb638b80e2d67 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:12.817250 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3e944-138b-4a98-b46c-62650115a1c9 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:12.817534 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3e944-138b-4a98-b46c-62650115a1c9 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:12.817807 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3e944-138b-4a98-b46c-62650115a1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:12.841365 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-b7c3e944-138b-4a98-b46c-62650115a1c9 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:12.842604 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b7c3e944-138b-4a98-b46c-62650115a1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:12.848015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ff31b-0ac4-4793-af9d-540f33592644 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:12.895024 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c33b12-7516-4114-ba61-b3afe5fa6c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:12.896106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c33b12-7516-4114-ba61-b3afe5fa6c89 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:12.896587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c33b12-7516-4114-ba61-b3afe5fa6c89 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:12.897046 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c33b12-7516-4114-ba61-b3afe5fa6c89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:12.898201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c33b12-7516-4114-ba61-b3afe5fa6c89 None admin] RBAC: Adding query filter params (role.id=8b837ae2ed57479a93acb638b80e2d67, scope.project.id=2a5a64cc8aa04972bb464407cb0312b4, user.id=90985ec8c43a4d7895942731dc1d002c) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:12.898688 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c33b12-7516-4114-ba61-b3afe5fa6c89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:12.900527 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c33b12-7516-4114-ba61-b3afe5fa6c89 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:12.909069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 236/488] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:36:12 2022] GET /identity/v3/role_assignments?user.id=90985ec8c43a4d7895942731dc1d002c&scope.project.id=2a5a64cc8aa04972bb464407cb0312b4&role.id=8b837ae2ed57479a93acb638b80e2d67 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:12.910965 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41ddc6-c1a8-4eed-8bf9-5b188d0b7bd0 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:12.911508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41ddc6-c1a8-4eed-8bf9-5b188d0b7bd0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:12.912007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41ddc6-c1a8-4eed-8bf9-5b188d0b7bd0 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:12.918668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 237/489] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:12.920470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ecb45-135d-4841-b47a-b563e2c0d64b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:12.920994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ecb45-135d-4841-b47a-b563e2c0d64b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:12.921534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ecb45-135d-4841-b47a-b563e2c0d64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:12.927472 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 253/490] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:36:12.943414 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c51ecb45-135d-4841-b47a-b563e2c0d64b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:12.944983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c51ecb45-135d-4841-b47a-b563e2c0d64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:13.029798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 238/491] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:36:13.299244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3e944-138b-4a98-b46c-62650115a1c9 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:13.310726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:13.374551 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ecb45-135d-4841-b47a-b563e2c0d64b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:13.435433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3e944-138b-4a98-b46c-62650115a1c9 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:13.465885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:13.474876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280432b2-b159-4645-ba0f-1648e80f9f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:13.475524 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280432b2-b159-4645-ba0f-1648e80f9f32 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:13.475805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280432b2-b159-4645-ba0f-1648e80f9f32 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:13.476107 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280432b2-b159-4645-ba0f-1648e80f9f32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:13.497818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280432b2-b159-4645-ba0f-1648e80f9f32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-280432b2-b159-4645-ba0f-1648e80f9f32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:36:13.502858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:13.503768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:13.504601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:13.504601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:13.504601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:13.504601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:13.504601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:36:13.504601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:36:13.504601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:36:13.504601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:36:13.504601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:36:13.504601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:36:13.504601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:13.504601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:13.504601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 254/492] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:36:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:13.512138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280432b2-b159-4645-ba0f-1648e80f9f32 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:13.570024 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ecb45-135d-4841-b47a-b563e2c0d64b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:13.591539 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:13.634460 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280432b2-b159-4645-ba0f-1648e80f9f32 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:13.668485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19362ea1-1a23-4d2e-81f3-272341de289f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:13.669182 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19362ea1-1a23-4d2e-81f3-272341de289f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:13.669444 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19362ea1-1a23-4d2e-81f3-272341de289f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:13.669787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19362ea1-1a23-4d2e-81f3-272341de289f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:13.677898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f349ed-de48-4f9a-bdb7-5bf90495c7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:13.679273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f349ed-de48-4f9a-bdb7-5bf90495c7f8 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:13.679763 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f349ed-de48-4f9a-bdb7-5bf90495c7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:13.680362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f349ed-de48-4f9a-bdb7-5bf90495c7f8 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:13.681263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f349ed-de48-4f9a-bdb7-5bf90495c7f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:13.681770 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f349ed-de48-4f9a-bdb7-5bf90495c7f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:13.683417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f349ed-de48-4f9a-bdb7-5bf90495c7f8 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:13.694586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 255/493] 10.222.0.36 () {56 vars in 1103 bytes} [Sat Oct 15 13:36:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:13.701943 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f349ed-de48-4f9a-bdb7-5bf90495c7f8 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:13.708657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19362ea1-1a23-4d2e-81f3-272341de289f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-19362ea1-1a23-4d2e-81f3-272341de289f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:36:13.713564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:13.714661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:13.715453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:13.715453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:13.715453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:13.715453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:13.715453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:36:13.715453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:36:13.715453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:36:13.715453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:36:13.715453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:36:13.715453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:36:13.715453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:36:13.715453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:13.715453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 239/494] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:36:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:13.725001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19362ea1-1a23-4d2e-81f3-272341de289f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:13.891330 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f349ed-de48-4f9a-bdb7-5bf90495c7f8 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:13.924852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19362ea1-1a23-4d2e-81f3-272341de289f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:13.961980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:13.964918 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db183016-6f53-45b2-bdd5-a45cf64b87b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:13.966078 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db183016-6f53-45b2-bdd5-a45cf64b87b6 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:13.966585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db183016-6f53-45b2-bdd5-a45cf64b87b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:13.967109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db183016-6f53-45b2-bdd5-a45cf64b87b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:13.987923 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db183016-6f53-45b2-bdd5-a45cf64b87b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:13.990630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db183016-6f53-45b2-bdd5-a45cf64b87b6 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-db183016-6f53-45b2-bdd5-a45cf64b87b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:14.020117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:14.021263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:14.022148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:14.022148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:14.022148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:14.022331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:14.023603 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 256/495] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:36:14.031347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db183016-6f53-45b2-bdd5-a45cf64b87b6 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:14.062659 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5ae24-5732-4d0d-a0b4-6664080f26c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:14.063699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5ae24-5732-4d0d-a0b4-6664080f26c8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:14.064192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5ae24-5732-4d0d-a0b4-6664080f26c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:14.064721 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5ae24-5732-4d0d-a0b4-6664080f26c8 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:14.065636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5ae24-5732-4d0d-a0b4-6664080f26c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:14.066125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5ae24-5732-4d0d-a0b4-6664080f26c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:14.067744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5ae24-5732-4d0d-a0b4-6664080f26c8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:14.084185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 240/496] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:36:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:14.089345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5ae24-5732-4d0d-a0b4-6664080f26c8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:14.212846 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db183016-6f53-45b2-bdd5-a45cf64b87b6 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:14.250420 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1fbf7-61a0-4afa-8f3f-8d3514ccd4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:14.251503 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1fbf7-61a0-4afa-8f3f-8d3514ccd4da None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:14.252038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1fbf7-61a0-4afa-8f3f-8d3514ccd4da None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:14.252512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1fbf7-61a0-4afa-8f3f-8d3514ccd4da None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:14.253407 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1fbf7-61a0-4afa-8f3f-8d3514ccd4da None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:14.253930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1fbf7-61a0-4afa-8f3f-8d3514ccd4da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:14.255535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1fbf7-61a0-4afa-8f3f-8d3514ccd4da None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:14.269874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 257/497] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:14.275803 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1fbf7-61a0-4afa-8f3f-8d3514ccd4da None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:14.321965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:14.384991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1fbf7-61a0-4afa-8f3f-8d3514ccd4da None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:14.420400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5ae24-5732-4d0d-a0b4-6664080f26c8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:14.424993 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16eedd9-2ac2-4a2a-9243-323a48cda760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:14.425962 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16eedd9-2ac2-4a2a-9243-323a48cda760 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:14.426490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16eedd9-2ac2-4a2a-9243-323a48cda760 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:14.426898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16eedd9-2ac2-4a2a-9243-323a48cda760 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:14.436011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16eedd9-2ac2-4a2a-9243-323a48cda760 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:14.437873 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16eedd9-2ac2-4a2a-9243-323a48cda760 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-d16eedd9-2ac2-4a2a-9243-323a48cda760 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:14.452273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:14.453405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:14.454720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:14.454720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:14.454720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:14.454720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:14.454720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:14.454720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:14.454720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:14.454720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:14.454720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 13:36:14.454720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:14.457107 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 258/498] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:36:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:36:14.463252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16eedd9-2ac2-4a2a-9243-323a48cda760 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:14.550416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4a9c5-30d1-4629-88ce-6c9799a7b4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:14.551576 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4a9c5-30d1-4629-88ce-6c9799a7b4f2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:14.552018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4a9c5-30d1-4629-88ce-6c9799a7b4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:14.552451 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4a9c5-30d1-4629-88ce-6c9799a7b4f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:14.571916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16eedd9-2ac2-4a2a-9243-323a48cda760 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:14.574255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:14.586642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4a9c5-30d1-4629-88ce-6c9799a7b4f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:14.588356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4a9c5-30d1-4629-88ce-6c9799a7b4f2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:14.611899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90a4dc-a96f-40e5-81ef-ef1784775a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:14.612871 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90a4dc-a96f-40e5-81ef-ef1784775a2b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:14.613310 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90a4dc-a96f-40e5-81ef-ef1784775a2b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:14.613721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90a4dc-a96f-40e5-81ef-ef1784775a2b None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:14.614703 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90a4dc-a96f-40e5-81ef-ef1784775a2b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:14.615160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90a4dc-a96f-40e5-81ef-ef1784775a2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:14.616806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90a4dc-a96f-40e5-81ef-ef1784775a2b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-4df4a9c5-30d1-4629-88ce-6c9799a7b4f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:14.627824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:14.629026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:14.630592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:14.630592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:14.630592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:14.636860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 241/499] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 548 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:14.640732 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 259/500] 10.222.0.36 () {56 vars in 1135 bytes} [Sat Oct 15 13:36:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:14.649032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4a9c5-30d1-4629-88ce-6c9799a7b4f2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:14.650254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90a4dc-a96f-40e5-81ef-ef1784775a2b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:14.715964 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:14.852161 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4a9c5-30d1-4629-88ce-6c9799a7b4f2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:14.917699 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90a4dc-a96f-40e5-81ef-ef1784775a2b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:14.938633 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7d304-8c4c-453b-b89e-d9615a651cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:14.940331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7d304-8c4c-453b-b89e-d9615a651cfa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:14.940734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7d304-8c4c-453b-b89e-d9615a651cfa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:14.946921 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7d304-8c4c-453b-b89e-d9615a651cfa None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:14.946921 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7d304-8c4c-453b-b89e-d9615a651cfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:14.946921 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7d304-8c4c-453b-b89e-d9615a651cfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:14.946921 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7d304-8c4c-453b-b89e-d9615a651cfa None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:14.963119 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 242/501] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:14.969479 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7d304-8c4c-453b-b89e-d9615a651cfa None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:15.000005 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5a89d-f935-4793-901f-5ad3eefa504b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:15.000654 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5a89d-f935-4793-901f-5ad3eefa504b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:15.000963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5a89d-f935-4793-901f-5ad3eefa504b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:15.001273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5a89d-f935-4793-901f-5ad3eefa504b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:15.065551 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:15.154557 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5a89d-f935-4793-901f-5ad3eefa504b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:15.156891 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5a89d-f935-4793-901f-5ad3eefa504b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:15.170796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:15.178021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7d304-8c4c-453b-b89e-d9615a651cfa None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:15.253939 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae4589-881a-4aed-824d-bbb4a6c86fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:15.254557 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae4589-881a-4aed-824d-bbb4a6c86fb0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:15.254835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae4589-881a-4aed-824d-bbb4a6c86fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:15.255103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae4589-881a-4aed-824d-bbb4a6c86fb0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:15.280793 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae4589-881a-4aed-824d-bbb4a6c86fb0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:15.282349 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae4589-881a-4aed-824d-bbb4a6c86fb0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-bdae4589-881a-4aed-824d-bbb4a6c86fb0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:15.305573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:15.306666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:15.307578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:15.307578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:15.307578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:15.307578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:15.307578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:15.307578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:15.307578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:15.307578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:15.307578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 13:36:15.307578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:15.316125 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 260/502] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:36:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 15 13:36:15.316344 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 243/503] 10.222.0.36 () {56 vars in 1093 bytes} [Sat Oct 15 13:36:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:15.318288 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5a89d-f935-4793-901f-5ad3eefa504b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:15.324938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae4589-881a-4aed-824d-bbb4a6c86fb0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:15.388273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:15.425795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5a89d-f935-4793-901f-5ad3eefa504b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:15.464812 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc1e56-5ca8-471c-ae1a-6c1b523a2cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:15.465462 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc1e56-5ca8-471c-ae1a-6c1b523a2cba None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:15.465739 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc1e56-5ca8-471c-ae1a-6c1b523a2cba None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:15.466012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc1e56-5ca8-471c-ae1a-6c1b523a2cba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:15.477779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bc1e56-5ca8-471c-ae1a-6c1b523a2cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:15.479343 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bc1e56-5ca8-471c-ae1a-6c1b523a2cba None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-b9bc1e56-5ca8-471c-ae1a-6c1b523a2cba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:15.496422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:15.497507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:15.498370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:15.498370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:15.498370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:15.502512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 261/504] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:36:15.504008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc1e56-5ca8-471c-ae1a-6c1b523a2cba None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:15.568000 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae4589-881a-4aed-824d-bbb4a6c86fb0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:15.631824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc1e56-5ca8-471c-ae1a-6c1b523a2cba None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:15.659026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:15.665250 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a693b-22a5-4eb0-abb5-de64afa029ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:15.665890 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a693b-22a5-4eb0-abb5-de64afa029ec None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:15.666160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a693b-22a5-4eb0-abb5-de64afa029ec None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:15.666458 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a693b-22a5-4eb0-abb5-de64afa029ec None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:15.667516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a693b-22a5-4eb0-abb5-de64afa029ec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:15.668242 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a693b-22a5-4eb0-abb5-de64afa029ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:15.669757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a693b-22a5-4eb0-abb5-de64afa029ec None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:15.693177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 244/505] 10.222.0.36 () {56 vars in 1133 bytes} [Sat Oct 15 13:36:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:15.701142 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a693b-22a5-4eb0-abb5-de64afa029ec None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:15.714884 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1d6f8-7715-45f6-945c-2dbaff6c6766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:15.715853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1d6f8-7715-45f6-945c-2dbaff6c6766 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:15.716394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1d6f8-7715-45f6-945c-2dbaff6c6766 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:15.716890 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1d6f8-7715-45f6-945c-2dbaff6c6766 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:15.717965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1d6f8-7715-45f6-945c-2dbaff6c6766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:15.718517 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1d6f8-7715-45f6-945c-2dbaff6c6766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:15.721266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1d6f8-7715-45f6-945c-2dbaff6c6766 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:15.748054 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 262/506] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:15.748669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:15.758552 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1d6f8-7715-45f6-945c-2dbaff6c6766 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:15.925791 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a693b-22a5-4eb0-abb5-de64afa029ec None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:15.963189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1d6f8-7715-45f6-945c-2dbaff6c6766 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:15.992240 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40b7ac-0baa-4d15-bb8e-1c5817159aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:15.993311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40b7ac-0baa-4d15-bb8e-1c5817159aad None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:15.993789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40b7ac-0baa-4d15-bb8e-1c5817159aad None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:15.994260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40b7ac-0baa-4d15-bb8e-1c5817159aad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:16.011687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40b7ac-0baa-4d15-bb8e-1c5817159aad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:16.013153 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40b7ac-0baa-4d15-bb8e-1c5817159aad None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:16.027606 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0dbe0-6728-4103-84ee-009b4515c722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:16.028742 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0dbe0-6728-4103-84ee-009b4515c722 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:16.029216 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0dbe0-6728-4103-84ee-009b4515c722 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:16.029693 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0dbe0-6728-4103-84ee-009b4515c722 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-7c40b7ac-0baa-4d15-bb8e-1c5817159aad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:16.033035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:16.034322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:16.035323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:16.035323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:16.035323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:16.035323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 245/507] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:16.035323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a3f8d-96dc-4247-935d-e6e053ad3731 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:16.035651 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a3f8d-96dc-4247-935d-e6e053ad3731 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:16.036431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a3f8d-96dc-4247-935d-e6e053ad3731 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:16.037506 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 246/508] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:16.038466 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea627b8-b361-40a2-9e2c-18c60bc8bcd6 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:16.043934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea627b8-b361-40a2-9e2c-18c60bc8bcd6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:16.043934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea627b8-b361-40a2-9e2c-18c60bc8bcd6 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:16.043934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 247/509] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:16.046346 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea627b8-b361-40a2-9e2c-18c60bc8bcd6 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:16.050958 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0dbe0-6728-4103-84ee-009b4515c722 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:16.052535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0dbe0-6728-4103-84ee-009b4515c722 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-f9d0dbe0-6728-4103-84ee-009b4515c722 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:16.064694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:16.066433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:16.068829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:16.068829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:16.068829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:16.068829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:16.068829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:16.068829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:16.068829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:16.068829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:16.068829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 263/510] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:16.068829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdbadf-aba1-44d1-8ac9-00aa8bbf59e3 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:16.068829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdbadf-aba1-44d1-8ac9-00aa8bbf59e3 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:16.069965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdbadf-aba1-44d1-8ac9-00aa8bbf59e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:16.091181 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-61cdbadf-aba1-44d1-8ac9-00aa8bbf59e3 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:16.092429 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-61cdbadf-aba1-44d1-8ac9-00aa8bbf59e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:16.124796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:16.175799 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 264/511] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:36:16.176819 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425f251-4f79-4f51-bdae-5d19637c00ba None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:16.177100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425f251-4f79-4f51-bdae-5d19637c00ba None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:16.177525 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425f251-4f79-4f51-bdae-5d19637c00ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:16.188362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:16.198487 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-2425f251-4f79-4f51-bdae-5d19637c00ba None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:16.199679 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2425f251-4f79-4f51-bdae-5d19637c00ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:16.281544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 265/512] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:36:16.282763 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425f251-4f79-4f51-bdae-5d19637c00ba None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:16.290486 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea627b8-b361-40a2-9e2c-18c60bc8bcd6 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:16.385624 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d7560-80ba-4269-9cd3-d887dde0753b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:16.387531 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d7560-80ba-4269-9cd3-d887dde0753b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:16.388288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d7560-80ba-4269-9cd3-d887dde0753b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:16.388595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d7560-80ba-4269-9cd3-d887dde0753b None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:16.389330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d7560-80ba-4269-9cd3-d887dde0753b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:16.389691 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d7560-80ba-4269-9cd3-d887dde0753b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:16.391190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d7560-80ba-4269-9cd3-d887dde0753b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:16.403092 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:16.416129 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425f251-4f79-4f51-bdae-5d19637c00ba None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:16.418501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 248/513] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:16.426246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d7560-80ba-4269-9cd3-d887dde0753b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:16.497303 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19479d3e-3b63-41d8-9769-dc382f5c67a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:16.497926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19479d3e-3b63-41d8-9769-dc382f5c67a7 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:16.498213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19479d3e-3b63-41d8-9769-dc382f5c67a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:16.498456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19479d3e-3b63-41d8-9769-dc382f5c67a7 None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:16.499201 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19479d3e-3b63-41d8-9769-dc382f5c67a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:16.499502 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19479d3e-3b63-41d8-9769-dc382f5c67a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:16.501054 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19479d3e-3b63-41d8-9769-dc382f5c67a7 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:16.536038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 266/514] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:16.551062 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19479d3e-3b63-41d8-9769-dc382f5c67a7 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:16.647927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d7560-80ba-4269-9cd3-d887dde0753b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:16.715005 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:16.730262 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651bf23-711b-483b-8f15-08da9a3d6e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:16.731310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651bf23-711b-483b-8f15-08da9a3d6e23 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:16.731756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651bf23-711b-483b-8f15-08da9a3d6e23 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:16.732239 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651bf23-711b-483b-8f15-08da9a3d6e23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:16.746260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651bf23-711b-483b-8f15-08da9a3d6e23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:16.748744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651bf23-711b-483b-8f15-08da9a3d6e23 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:16.764794 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19479d3e-3b63-41d8-9769-dc382f5c67a7 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-e651bf23-711b-483b-8f15-08da9a3d6e23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:16.771303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:16.773294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:16.774982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:16.774982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:16.774982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:16.774982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:16.774982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:16.774982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:16.774982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:16.774982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:16.781537 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 249/515] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:36:16.784644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd9cf8-b824-40ed-84d8-437dee89c7ad None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:16.785111 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd9cf8-b824-40ed-84d8-437dee89c7ad None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:16.785588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd9cf8-b824-40ed-84d8-437dee89c7ad None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:16.788604 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 250/516] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:16.789818 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd9cf8-b824-40ed-84d8-437dee89c7ad None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:16.827183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be363b-0d08-4168-a2b3-9152ac36931f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:16.828124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be363b-0d08-4168-a2b3-9152ac36931f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:16.828568 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be363b-0d08-4168-a2b3-9152ac36931f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:16.828977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be363b-0d08-4168-a2b3-9152ac36931f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:16.830921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be363b-0d08-4168-a2b3-9152ac36931f None admin] RBAC: Adding query filter params (role.id=8b837ae2ed57479a93acb638b80e2d67, scope.project.id=2a5a64cc8aa04972bb464407cb0312b4, user.id=d011a40951564323bf7a9c5601df1470) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:16.831394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be363b-0d08-4168-a2b3-9152ac36931f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:16.833429 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be363b-0d08-4168-a2b3-9152ac36931f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:16.844362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 267/517] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:36:16 2022] GET /identity/v3/role_assignments?user.id=d011a40951564323bf7a9c5601df1470&scope.project.id=2a5a64cc8aa04972bb464407cb0312b4&role.id=8b837ae2ed57479a93acb638b80e2d67 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:16.845943 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be363b-0d08-4168-a2b3-9152ac36931f None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:16.876425 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:16.969117 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd9cf8-b824-40ed-84d8-437dee89c7ad None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:17.019272 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1516db-bb8a-41d6-96dc-1483da606c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:17.020264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1516db-bb8a-41d6-96dc-1483da606c0e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:17.020603 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1516db-bb8a-41d6-96dc-1483da606c0e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:17.020840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1516db-bb8a-41d6-96dc-1483da606c0e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:17.021568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1516db-bb8a-41d6-96dc-1483da606c0e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:17.024007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1516db-bb8a-41d6-96dc-1483da606c0e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:17.029660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be363b-0d08-4168-a2b3-9152ac36931f None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-4d1516db-bb8a-41d6-96dc-1483da606c0e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:17.043154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:17.045328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:17.052236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 251/518] 10.222.0.36 () {56 vars in 1129 bytes} [Sat Oct 15 13:36:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:17.053077 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e977c-fe11-48ae-a4f3-ca0c07cb0314 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:17.053077 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e977c-fe11-48ae-a4f3-ca0c07cb0314 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:17.053077 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e977c-fe11-48ae-a4f3-ca0c07cb0314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:17.078293 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5d5e977c-fe11-48ae-a4f3-ca0c07cb0314 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:17.080335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5d5e977c-fe11-48ae-a4f3-ca0c07cb0314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:17.096057 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ce89d-6ead-4a65-9d74-8071485f77ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:17.096739 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ce89d-6ead-4a65-9d74-8071485f77ba None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:17.097476 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ce89d-6ead-4a65-9d74-8071485f77ba None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:17.097771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ce89d-6ead-4a65-9d74-8071485f77ba None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:17.098488 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ce89d-6ead-4a65-9d74-8071485f77ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:17.098789 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ce89d-6ead-4a65-9d74-8071485f77ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:17.100644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ce89d-6ead-4a65-9d74-8071485f77ba None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:17.115993 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 268/519] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:17.116984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ce89d-6ead-4a65-9d74-8071485f77ba None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:17.154764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:17.183477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 252/520] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:36:17.184013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e977c-fe11-48ae-a4f3-ca0c07cb0314 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:17.265870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:17.321329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e977c-fe11-48ae-a4f3-ca0c07cb0314 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:17.332358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ce89d-6ead-4a65-9d74-8071485f77ba None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:17.365428 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c7b60-2bc9-4332-a7ea-25d2ee20f4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:17.366440 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c7b60-2bc9-4332-a7ea-25d2ee20f4e4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:17.366938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c7b60-2bc9-4332-a7ea-25d2ee20f4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:17.367407 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c7b60-2bc9-4332-a7ea-25d2ee20f4e4 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:17.368453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c7b60-2bc9-4332-a7ea-25d2ee20f4e4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:17.368942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c7b60-2bc9-4332-a7ea-25d2ee20f4e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:17.370757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c7b60-2bc9-4332-a7ea-25d2ee20f4e4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:17.377085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 253/521] 10.222.0.36 () {56 vars in 1124 bytes} [Sat Oct 15 13:36:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:17.378443 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c7b60-2bc9-4332-a7ea-25d2ee20f4e4 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:17.404963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b5787-4e96-473c-83a8-fd55a7576f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:17.406105 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b5787-4e96-473c-83a8-fd55a7576f7e None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:17.406569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b5787-4e96-473c-83a8-fd55a7576f7e None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:17.407075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b5787-4e96-473c-83a8-fd55a7576f7e None admin] PATH_INFO: `/v3/services/network` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:17.408141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b5787-4e96-473c-83a8-fd55a7576f7e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:17.410044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b5787-4e96-473c-83a8-fd55a7576f7e None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-cf1b5787-4e96-473c-83a8-fd55a7576f7e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:17.422712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:17.423896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:17.424733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:17.424733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:17.424733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:17.424733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:17.424733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:17.424733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:17.424733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:17.424733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:17.424733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:17.424733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:17.424733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:17.424733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:17.424733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 13:36:17.424733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:17.427432 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 269/522] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:17 2022] GET /identity/v3/services/network => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:17.428561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b5787-4e96-473c-83a8-fd55a7576f7e None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:17.482648 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:17.592401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c7b60-2bc9-4332-a7ea-25d2ee20f4e4 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:17.608878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b5787-4e96-473c-83a8-fd55a7576f7e None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:17.647834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1eaba7-65c6-4816-ad68-85e757a21592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:17.649023 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1eaba7-65c6-4816-ad68-85e757a21592 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:17.649524 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1eaba7-65c6-4816-ad68-85e757a21592 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:17.650010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1eaba7-65c6-4816-ad68-85e757a21592 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:17.650926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1eaba7-65c6-4816-ad68-85e757a21592 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:17.651454 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1eaba7-65c6-4816-ad68-85e757a21592 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:17.653101 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1eaba7-65c6-4816-ad68-85e757a21592 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:17.658271 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cca8f5-c674-4f85-b32c-68bfbd574108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:17.659523 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cca8f5-c674-4f85-b32c-68bfbd574108 None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:17.660071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cca8f5-c674-4f85-b32c-68bfbd574108 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:17.660554 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cca8f5-c674-4f85-b32c-68bfbd574108 None admin] PATH_INFO: `/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/2e13692c1e83413a909d99bb7db2e4b8/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:17.661202 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 270/523] 10.222.0.36 () {56 vars in 1124 bytes} [Sat Oct 15 13:36:17 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:17.662268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1eaba7-65c6-4816-ad68-85e757a21592 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:17.664240 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cca8f5-c674-4f85-b32c-68bfbd574108 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a5a64cc8aa04972bb464407cb0312b4, user_id=2e13692c1e83413a909d99bb7db2e4b8, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:17.667332 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cca8f5-c674-4f85-b32c-68bfbd574108 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:17.668504 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:17.696250 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 254/524] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:36:17 2022] PUT /identity/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/2e13692c1e83413a909d99bb7db2e4b8/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:36:17.698330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cca8f5-c674-4f85-b32c-68bfbd574108 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:17.814085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1eaba7-65c6-4816-ad68-85e757a21592 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:17.868335 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb076df-3903-48b3-bc87-2f1fb5554424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:17.869448 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb076df-3903-48b3-bc87-2f1fb5554424 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:17.878548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb076df-3903-48b3-bc87-2f1fb5554424 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:17.878990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb076df-3903-48b3-bc87-2f1fb5554424 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:17.880001 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb076df-3903-48b3-bc87-2f1fb5554424 None admin] RBAC: Adding query filter params (name=network) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:17.880497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb076df-3903-48b3-bc87-2f1fb5554424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:17.883308 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb076df-3903-48b3-bc87-2f1fb5554424 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:17.900158 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 271/525] 10.222.0.36 () {56 vars in 1112 bytes} [Sat Oct 15 13:36:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:17.913271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb076df-3903-48b3-bc87-2f1fb5554424 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:17.918860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cca8f5-c674-4f85-b32c-68bfbd574108 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:17.939469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:17.992369 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bc071-38eb-4613-b62b-9895f0652f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:17.994059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bc071-38eb-4613-b62b-9895f0652f77 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:17.996129 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bc071-38eb-4613-b62b-9895f0652f77 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:17.997028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bc071-38eb-4613-b62b-9895f0652f77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:18.018181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bc071-38eb-4613-b62b-9895f0652f77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:18.021168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bc071-38eb-4613-b62b-9895f0652f77 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:18.027764 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-cc8bc071-38eb-4613-b62b-9895f0652f77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:18.051722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:18.053904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:18.055700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:18.055700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:18.055700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:18.063961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 255/526] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:18.066468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bc071-38eb-4613-b62b-9895f0652f77 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:18.096517 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb076df-3903-48b3-bc87-2f1fb5554424 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:18.177207 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed59c26-dd20-4569-9712-719adff6c67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:18.178730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed59c26-dd20-4569-9712-719adff6c67f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:18.179486 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed59c26-dd20-4569-9712-719adff6c67f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:18.180240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed59c26-dd20-4569-9712-719adff6c67f None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:18.181626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed59c26-dd20-4569-9712-719adff6c67f None admin] RBAC: Adding query filter params (type=network) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:18.182418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed59c26-dd20-4569-9712-719adff6c67f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:18.185133 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed59c26-dd20-4569-9712-719adff6c67f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:18.195162 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 272/527] 10.222.0.36 () {56 vars in 1112 bytes} [Sat Oct 15 13:36:17 2022] GET /identity/v3/services?type=network => generated 124 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:18.226103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bc071-38eb-4613-b62b-9895f0652f77 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:18.272155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a2190-3cf3-4bd8-bbc2-a12f515aad93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:18.272799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a2190-3cf3-4bd8-bbc2-a12f515aad93 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:18.273067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a2190-3cf3-4bd8-bbc2-a12f515aad93 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:18.273335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a2190-3cf3-4bd8-bbc2-a12f515aad93 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:18.273998 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a2190-3cf3-4bd8-bbc2-a12f515aad93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:18.274298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a2190-3cf3-4bd8-bbc2-a12f515aad93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:18.275744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a2190-3cf3-4bd8-bbc2-a12f515aad93 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:18.290385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 256/528] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:36:18.299209 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed59c26-dd20-4569-9712-719adff6c67f None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:18.457279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed59c26-dd20-4569-9712-719adff6c67f None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:18.492899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:18.504436 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20864573-504e-4155-a348-13d91fcc3938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:18.505077 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20864573-504e-4155-a348-13d91fcc3938 None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:18.505468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20864573-504e-4155-a348-13d91fcc3938 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:18.505745 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20864573-504e-4155-a348-13d91fcc3938 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:18.506428 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20864573-504e-4155-a348-13d91fcc3938 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:18.507897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20864573-504e-4155-a348-13d91fcc3938 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:18.538980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 273/529] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:36:18 2022] POST /identity/v3/users => generated 254 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:36:20.120898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0669b-3d46-4c33-85b1-450e63e7534e None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:20.121952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0669b-3d46-4c33-85b1-450e63e7534e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:20.122501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0669b-3d46-4c33-85b1-450e63e7534e None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:20.124432 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 257/530] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:20.131937 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06943c3d-b7b1-4681-a8d9-ebb5acecda4c None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:20.132874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06943c3d-b7b1-4681-a8d9-ebb5acecda4c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:20.133347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06943c3d-b7b1-4681-a8d9-ebb5acecda4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:20.155761 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-06943c3d-b7b1-4681-a8d9-ebb5acecda4c None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:20.157509 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-06943c3d-b7b1-4681-a8d9-ebb5acecda4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:20.235780 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 274/531] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:36:20.714594 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0669b-3d46-4c33-85b1-450e63e7534e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:20.744769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:20.833001 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18b0f4-f687-42f7-b34f-3cbeb92814a1 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:20.833458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18b0f4-f687-42f7-b34f-3cbeb92814a1 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:20.833923 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18b0f4-f687-42f7-b34f-3cbeb92814a1 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:20.836395 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 275/532] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:20.843672 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bb361-d287-40f3-9e43-94da137f01a9 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:20.844151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bb361-d287-40f3-9e43-94da137f01a9 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:20.844618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bb361-d287-40f3-9e43-94da137f01a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:20.898690 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-2f4bb361-d287-40f3-9e43-94da137f01a9 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:20.901116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2f4bb361-d287-40f3-9e43-94da137f01a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:20.973309 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0669b-3d46-4c33-85b1-450e63e7534e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:21.047112 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 276/533] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:36:21.049095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab59e2-ef17-445e-ba44-f17c0b99f252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:21.049409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f159ec8-f124-4744-a83f-0aaf48cf0d4c None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:21.049888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f159ec8-f124-4744-a83f-0aaf48cf0d4c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:21.050364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f159ec8-f124-4744-a83f-0aaf48cf0d4c None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:21.051000 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab59e2-ef17-445e-ba44-f17c0b99f252 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:21.051668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab59e2-ef17-445e-ba44-f17c0b99f252 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:21.052376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab59e2-ef17-445e-ba44-f17c0b99f252 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:21.052722 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 277/534] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:21.061421 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058068-7d1d-4f94-b8ea-8f4fa454cc4e None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:21.061908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058068-7d1d-4f94-b8ea-8f4fa454cc4e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:21.062376 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058068-7d1d-4f94-b8ea-8f4fa454cc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:21.088880 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab59e2-ef17-445e-ba44-f17c0b99f252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-44ab59e2-ef17-445e-ba44-f17c0b99f252 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:36:21.093797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:21.096041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:21.097852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:21.097852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:21.097852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:21.097852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:21.097852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:36:21.097852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:36:21.097852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:36:21.097852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:36:21.097852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:36:21.097852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:36:21.097852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:36:21.097852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:21.097852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 258/535] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:36:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:21.103554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-4a058068-7d1d-4f94-b8ea-8f4fa454cc4e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:21.103554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4a058068-7d1d-4f94-b8ea-8f4fa454cc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:21.104887 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab59e2-ef17-445e-ba44-f17c0b99f252 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:21.152466 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:21.190566 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 278/536] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:36:21.262927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:21.292189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab59e2-ef17-445e-ba44-f17c0b99f252 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:21.340828 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2d536-d51e-4eb3-b1ff-f12f5a5712f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:21.341452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2d536-d51e-4eb3-b1ff-f12f5a5712f1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:21.341727 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2d536-d51e-4eb3-b1ff-f12f5a5712f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:21.341995 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2d536-d51e-4eb3-b1ff-f12f5a5712f1 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:21.342667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2d536-d51e-4eb3-b1ff-f12f5a5712f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:21.342958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2d536-d51e-4eb3-b1ff-f12f5a5712f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:21.344395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2d536-d51e-4eb3-b1ff-f12f5a5712f1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:21.358115 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 259/537] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:36:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:21.364400 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a058068-7d1d-4f94-b8ea-8f4fa454cc4e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:21.497872 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a058068-7d1d-4f94-b8ea-8f4fa454cc4e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:21.551774 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09948e-9fb1-4802-8497-0458d946e6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:21.552346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09948e-9fb1-4802-8497-0458d946e6be None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:21.552625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09948e-9fb1-4802-8497-0458d946e6be None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:21.552896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09948e-9fb1-4802-8497-0458d946e6be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:21.571374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09948e-9fb1-4802-8497-0458d946e6be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:21.572894 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09948e-9fb1-4802-8497-0458d946e6be None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-3f09948e-9fb1-4802-8497-0458d946e6be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:21.591750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:21.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:21.596593 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:21.596593 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:21.596593 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:21.596593 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 279/538] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:21.601938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2d536-d51e-4eb3-b1ff-f12f5a5712f1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:21.663810 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09948e-9fb1-4802-8497-0458d946e6be None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:21.761781 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:21.785317 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2d536-d51e-4eb3-b1ff-f12f5a5712f1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:21.823192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433035e-4cc5-4189-ae7b-eb0e8f7e2836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:21.823723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433035e-4cc5-4189-ae7b-eb0e8f7e2836 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:21.824037 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433035e-4cc5-4189-ae7b-eb0e8f7e2836 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:21.824309 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433035e-4cc5-4189-ae7b-eb0e8f7e2836 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:21.825025 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433035e-4cc5-4189-ae7b-eb0e8f7e2836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:21.825332 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433035e-4cc5-4189-ae7b-eb0e8f7e2836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:21.826726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433035e-4cc5-4189-ae7b-eb0e8f7e2836 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:21.839106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:21.851728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 260/539] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:36:21.853306 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433035e-4cc5-4189-ae7b-eb0e8f7e2836 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:21.878068 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09948e-9fb1-4802-8497-0458d946e6be None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:21.924457 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3789af6-bd22-4691-aabc-30be8bbfe38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:21.925511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3789af6-bd22-4691-aabc-30be8bbfe38b None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:21.926004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3789af6-bd22-4691-aabc-30be8bbfe38b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:21.926487 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3789af6-bd22-4691-aabc-30be8bbfe38b None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:21.927740 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3789af6-bd22-4691-aabc-30be8bbfe38b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:21.930419 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3789af6-bd22-4691-aabc-30be8bbfe38b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:21.955009 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 280/540] 10.222.0.36 () {60 vars in 1141 bytes} [Sat Oct 15 13:36:21 2022] POST /identity/v3/services => generated 251 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:36:21.956842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3505e-0a23-44eb-adba-44c6afc246f6 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:21.957262 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3505e-0a23-44eb-adba-44c6afc246f6 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:21.957686 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3505e-0a23-44eb-adba-44c6afc246f6 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:21.963935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 281/541] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:21.963935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3505e-0a23-44eb-adba-44c6afc246f6 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:21.978355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433035e-4cc5-4189-ae7b-eb0e8f7e2836 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:22.050081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a3c84-b6da-4572-b419-ec63348de35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:22.051681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a3c84-b6da-4572-b419-ec63348de35c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:22.052168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a3c84-b6da-4572-b419-ec63348de35c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:22.052617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a3c84-b6da-4572-b419-ec63348de35c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:22.069788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a3c84-b6da-4572-b419-ec63348de35c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-bc9a3c84-b6da-4572-b419-ec63348de35c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:36:22.078498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:22.080627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:22.082323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:22.082323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:22.082323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:22.082323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:22.082323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:36:22.082323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:36:22.082323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:36:22.082323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:36:22.082323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:36:22.082323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:36:22.082323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:36:22.082323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:22.082323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 261/542] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:36:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:22.083745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37d990-adc4-465c-b129-835f0455f75e None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:22.083745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37d990-adc4-465c-b129-835f0455f75e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:22.083745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37d990-adc4-465c-b129-835f0455f75e None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:22.087781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 262/543] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:22.089538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998d7e4-ab7c-4bb4-a14c-d8ee50e1d45a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:22.089930 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998d7e4-ab7c-4bb4-a14c-d8ee50e1d45a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:22.090303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998d7e4-ab7c-4bb4-a14c-d8ee50e1d45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:22.113641 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a998d7e4-ab7c-4bb4-a14c-d8ee50e1d45a None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:22.115729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a998d7e4-ab7c-4bb4-a14c-d8ee50e1d45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:22.130235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:22.149857 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3505e-0a23-44eb-adba-44c6afc246f6 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:22.203625 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:22.209331 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d06dbe-acec-4ad7-bdef-f647e43e3e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:22.209904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d06dbe-acec-4ad7-bdef-f647e43e3e9d None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:22.209990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d06dbe-acec-4ad7-bdef-f647e43e3e9d None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:22.210358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d06dbe-acec-4ad7-bdef-f647e43e3e9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:22.224632 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d06dbe-acec-4ad7-bdef-f647e43e3e9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:22.227327 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d06dbe-acec-4ad7-bdef-f647e43e3e9d None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:22.232909 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 263/544] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:22 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:36:22.234678 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998d7e4-ab7c-4bb4-a14c-d8ee50e1d45a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-19d06dbe-acec-4ad7-bdef-f647e43e3e9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:22.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:22.245998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:22.248701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:22.248701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:22.248701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:22.248701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:22.248701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:22.248701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:22.248701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:22.248701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:22.248701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 13:36:22.248701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:22.248701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 282/545] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:36:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:22.248701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd8ed8-8e4d-4df0-b8cf-9a6f6c1052be None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:22.248701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd8ed8-8e4d-4df0-b8cf-9a6f6c1052be None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:22.250090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd8ed8-8e4d-4df0-b8cf-9a6f6c1052be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:22.277732 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-75fd8ed8-8e4d-4df0-b8cf-9a6f6c1052be None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:22.279554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-75fd8ed8-8e4d-4df0-b8cf-9a6f6c1052be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:22.418454 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 283/546] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:22 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 15 13:36:22.420936 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd8ed8-8e4d-4df0-b8cf-9a6f6c1052be None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:22.460517 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998d7e4-ab7c-4bb4-a14c-d8ee50e1d45a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:22.523638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ec6ad-39f9-48e2-8ad1-9b074f37625e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:22.524630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ec6ad-39f9-48e2-8ad1-9b074f37625e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:22.525064 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ec6ad-39f9-48e2-8ad1-9b074f37625e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:22.525505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ec6ad-39f9-48e2-8ad1-9b074f37625e None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:22.526638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ec6ad-39f9-48e2-8ad1-9b074f37625e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:22.527116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ec6ad-39f9-48e2-8ad1-9b074f37625e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:22.529656 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ec6ad-39f9-48e2-8ad1-9b074f37625e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:22.554018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 264/547] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:36:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:22.559294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ec6ad-39f9-48e2-8ad1-9b074f37625e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:22.679170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:22.724206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:22.743859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd8ed8-8e4d-4df0-b8cf-9a6f6c1052be None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:22.795673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ec6ad-39f9-48e2-8ad1-9b074f37625e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:22.799322 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8665d60-2ff3-4cf9-bad0-ea2035145aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:22.800628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8665d60-2ff3-4cf9-bad0-ea2035145aac None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:22.800904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8665d60-2ff3-4cf9-bad0-ea2035145aac None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:22.801167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8665d60-2ff3-4cf9-bad0-ea2035145aac None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:22.801908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8665d60-2ff3-4cf9-bad0-ea2035145aac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:22.802208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8665d60-2ff3-4cf9-bad0-ea2035145aac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:22.803656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8665d60-2ff3-4cf9-bad0-ea2035145aac None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:22.829175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 284/548] 10.222.0.36 () {56 vars in 1135 bytes} [Sat Oct 15 13:36:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:22.829175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8665d60-2ff3-4cf9-bad0-ea2035145aac None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:22.868361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58dfc34-a9ea-4094-87dc-8d3b778e58f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:22.869427 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58dfc34-a9ea-4094-87dc-8d3b778e58f4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:22.869913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58dfc34-a9ea-4094-87dc-8d3b778e58f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:22.870376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58dfc34-a9ea-4094-87dc-8d3b778e58f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:22.894370 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58dfc34-a9ea-4094-87dc-8d3b778e58f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:22.897144 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58dfc34-a9ea-4094-87dc-8d3b778e58f4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f58dfc34-a9ea-4094-87dc-8d3b778e58f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:22.924670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:22.925729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:22.926599 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:22.926599 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:22.926599 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:22.926768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 265/549] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:22.928274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58dfc34-a9ea-4094-87dc-8d3b778e58f4 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:22.992827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8665d60-2ff3-4cf9-bad0-ea2035145aac None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:23.056278 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d21467-2de8-4352-98e2-88b81da341c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:23.058150 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d21467-2de8-4352-98e2-88b81da341c3 None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:23.058795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d21467-2de8-4352-98e2-88b81da341c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:23.059356 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d21467-2de8-4352-98e2-88b81da341c3 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:23.061342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d21467-2de8-4352-98e2-88b81da341c3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:23.063962 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d21467-2de8-4352-98e2-88b81da341c3 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:23.082606 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:23.094389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 285/550] 10.222.0.36 () {60 vars in 1141 bytes} [Sat Oct 15 13:36:22 2022] POST /identity/v3/services => generated 240 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:36:23.095560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:23.096104 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d21467-2de8-4352-98e2-88b81da341c3 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:23.112688 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58dfc34-a9ea-4094-87dc-8d3b778e58f4 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:23.224210 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756620b3-bd06-4747-bfb9-92dc4c430839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:23.225879 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756620b3-bd06-4747-bfb9-92dc4c430839 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:23.227056 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756620b3-bd06-4747-bfb9-92dc4c430839 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:23.227837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756620b3-bd06-4747-bfb9-92dc4c430839 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:23.254577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756620b3-bd06-4747-bfb9-92dc4c430839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-756620b3-bd06-4747-bfb9-92dc4c430839 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:36:23.265260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:23.267313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:23.269536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:23.269536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:23.269536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:23.269536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:23.269536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:36:23.269536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:36:23.269536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:36:23.269536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:36:23.269536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:36:23.269536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:36:23.269536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:23.269536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:23.272743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 266/551] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:36:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:23.274387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756620b3-bd06-4747-bfb9-92dc4c430839 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:23.338992 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d21467-2de8-4352-98e2-88b81da341c3 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:23.416179 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355fc4a-8233-4dc5-830c-3220e28ef58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:23.417842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355fc4a-8233-4dc5-830c-3220e28ef58c None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:23.418671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355fc4a-8233-4dc5-830c-3220e28ef58c None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:23.419161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355fc4a-8233-4dc5-830c-3220e28ef58c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:23.447514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756620b3-bd06-4747-bfb9-92dc4c430839 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:23.450334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355fc4a-8233-4dc5-830c-3220e28ef58c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:23.454887 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355fc4a-8233-4dc5-830c-3220e28ef58c None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:23.493032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e9517-7a17-47bd-889e-9eec1e09192b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:23.493886 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e9517-7a17-47bd-889e-9eec1e09192b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:23.494026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e9517-7a17-47bd-889e-9eec1e09192b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:23.494373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e9517-7a17-47bd-889e-9eec1e09192b None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-5355fc4a-8233-4dc5-830c-3220e28ef58c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:23.494519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:23.496225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:23.497864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:23.497864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:23.497864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:23.498192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e9517-7a17-47bd-889e-9eec1e09192b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:23.498192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e9517-7a17-47bd-889e-9eec1e09192b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:23.498443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 286/552] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:23.498567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e9517-7a17-47bd-889e-9eec1e09192b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:23.498683 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355fc4a-8233-4dc5-830c-3220e28ef58c None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:23.514386 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 267/553] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:23.520126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e9517-7a17-47bd-889e-9eec1e09192b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:23.542976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:23.582136 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:23.680729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e9517-7a17-47bd-889e-9eec1e09192b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:23.699922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355fc4a-8233-4dc5-830c-3220e28ef58c None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:23.723133 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896613d3-7870-49cf-83e1-9949efae5aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:23.724281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896613d3-7870-49cf-83e1-9949efae5aaa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:23.724761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896613d3-7870-49cf-83e1-9949efae5aaa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:23.725264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896613d3-7870-49cf-83e1-9949efae5aaa None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:23.726182 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896613d3-7870-49cf-83e1-9949efae5aaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:23.726701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896613d3-7870-49cf-83e1-9949efae5aaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:23.728398 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896613d3-7870-49cf-83e1-9949efae5aaa None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:23.755614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1254d2d-b6b4-43c3-bb6b-6b73d43c1675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:23.756568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 268/554] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:23.756877 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1254d2d-b6b4-43c3-bb6b-6b73d43c1675 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:23.757355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1254d2d-b6b4-43c3-bb6b-6b73d43c1675 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:23.757870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1254d2d-b6b4-43c3-bb6b-6b73d43c1675 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:23.758068 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896613d3-7870-49cf-83e1-9949efae5aaa None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:23.758945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1254d2d-b6b4-43c3-bb6b-6b73d43c1675 None admin] RBAC: Adding query filter params (name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:23.759453 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1254d2d-b6b4-43c3-bb6b-6b73d43c1675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:23.761111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1254d2d-b6b4-43c3-bb6b-6b73d43c1675 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:23.771931 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 287/555] 10.222.0.36 () {56 vars in 1103 bytes} [Sat Oct 15 13:36:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:23.773159 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1254d2d-b6b4-43c3-bb6b-6b73d43c1675 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:23.940048 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:23.958026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1254d2d-b6b4-43c3-bb6b-6b73d43c1675 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:23.986513 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896613d3-7870-49cf-83e1-9949efae5aaa None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:23.992359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:24.031989 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249de29b-a97b-4cb4-8525-badacff98e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:24.033758 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249de29b-a97b-4cb4-8525-badacff98e24 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:24.034236 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249de29b-a97b-4cb4-8525-badacff98e24 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:24.034685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249de29b-a97b-4cb4-8525-badacff98e24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:24.051431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249de29b-a97b-4cb4-8525-badacff98e24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:24.054430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249de29b-a97b-4cb4-8525-badacff98e24 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:24.055290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f5ed5-adab-44d2-bf1a-3b24ccf7a9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:24.056806 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f5ed5-adab-44d2-bf1a-3b24ccf7a9d1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:24.057585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f5ed5-adab-44d2-bf1a-3b24ccf7a9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:24.058333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f5ed5-adab-44d2-bf1a-3b24ccf7a9d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:24.074298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f5ed5-adab-44d2-bf1a-3b24ccf7a9d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:24.077300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f5ed5-adab-44d2-bf1a-3b24ccf7a9d1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-249de29b-a97b-4cb4-8525-badacff98e24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:24.079447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:24.081294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:24.084121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:24.084121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:24.084121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:24.084121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:24.084121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:24.084121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:24.084121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:24.084121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:24.084121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 288/556] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:24.085734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249de29b-a97b-4cb4-8525-badacff98e24 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f09f5ed5-adab-44d2-bf1a-3b24ccf7a9d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:24.108185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:24.110565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:24.112609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:24.112609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:24.112609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:24.112609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:24.112609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:24.112609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:24.112609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:24.112609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:24.112609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 13:36:24.112609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:24.124110 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 269/557] 10.222.0.36 () {56 vars in 1093 bytes} [Sat Oct 15 13:36:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:24.126175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f5ed5-adab-44d2-bf1a-3b24ccf7a9d1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:24.301908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249de29b-a97b-4cb4-8525-badacff98e24 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:24.336715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f5ed5-adab-44d2-bf1a-3b24ccf7a9d1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:24.371682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07defe87-dacc-4d52-a9dd-63d8f8e4601e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:24.373345 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07defe87-dacc-4d52-a9dd-63d8f8e4601e None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:24.373831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07defe87-dacc-4d52-a9dd-63d8f8e4601e None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:24.374289 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07defe87-dacc-4d52-a9dd-63d8f8e4601e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:24.398696 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07defe87-dacc-4d52-a9dd-63d8f8e4601e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:24.402025 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07defe87-dacc-4d52-a9dd-63d8f8e4601e None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:24.415425 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d508c-be7e-4750-966f-85f1dbe94696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:24.416382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d508c-be7e-4750-966f-85f1dbe94696 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:24.416789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d508c-be7e-4750-966f-85f1dbe94696 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:24.417192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d508c-be7e-4750-966f-85f1dbe94696 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:24.419186 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d508c-be7e-4750-966f-85f1dbe94696 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:24.419695 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d508c-be7e-4750-966f-85f1dbe94696 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:24.422325 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d508c-be7e-4750-966f-85f1dbe94696 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:24.428111 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-07defe87-dacc-4d52-a9dd-63d8f8e4601e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:24.437999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:24.440127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:24.441744 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:24.441744 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:24.441744 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:24.445266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 289/558] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:24.447503 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07defe87-dacc-4d52-a9dd-63d8f8e4601e None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:24.455845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 270/559] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:24.458174 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d508c-be7e-4750-966f-85f1dbe94696 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:24.507323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:24.656919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07defe87-dacc-4d52-a9dd-63d8f8e4601e None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:24.680564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d508c-be7e-4750-966f-85f1dbe94696 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:24.724756 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ecb61-f9e7-4b64-8755-059b40f14188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:24.727028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ecb61-f9e7-4b64-8755-059b40f14188 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:24.727822 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ecb61-f9e7-4b64-8755-059b40f14188 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:24.728709 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ecb61-f9e7-4b64-8755-059b40f14188 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:24.730412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ecb61-f9e7-4b64-8755-059b40f14188 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:24.732405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ecb61-f9e7-4b64-8755-059b40f14188 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:24.736915 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ecb61-f9e7-4b64-8755-059b40f14188 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:24.762273 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778755b-6e49-4de3-ae7e-68b0211e87a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:24.763261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778755b-6e49-4de3-ae7e-68b0211e87a7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:24.763706 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778755b-6e49-4de3-ae7e-68b0211e87a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:24.764199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778755b-6e49-4de3-ae7e-68b0211e87a7 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:24.765264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778755b-6e49-4de3-ae7e-68b0211e87a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:24.765740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778755b-6e49-4de3-ae7e-68b0211e87a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:24.768404 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778755b-6e49-4de3-ae7e-68b0211e87a7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:24.785142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 290/560] 10.222.0.36 () {56 vars in 1133 bytes} [Sat Oct 15 13:36:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:24.785843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ecb61-f9e7-4b64-8755-059b40f14188 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:24.798343 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 271/561] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:24.800491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778755b-6e49-4de3-ae7e-68b0211e87a7 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:24.926738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:24.981277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:25.025699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778755b-6e49-4de3-ae7e-68b0211e87a7 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:25.026810 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ecb61-f9e7-4b64-8755-059b40f14188 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:25.093142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8e0c6-bdd7-408d-b5b9-9b82c40a5187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:25.094285 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8e0c6-bdd7-408d-b5b9-9b82c40a5187 None admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:25.094744 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8e0c6-bdd7-408d-b5b9-9b82c40a5187 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:25.095203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8e0c6-bdd7-408d-b5b9-9b82c40a5187 None admin] PATH_INFO: `/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/d011a40951564323bf7a9c5601df1470/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:25.098644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8e0c6-bdd7-408d-b5b9-9b82c40a5187 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a5a64cc8aa04972bb464407cb0312b4, user_id=d011a40951564323bf7a9c5601df1470, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:25.101539 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8e0c6-bdd7-408d-b5b9-9b82c40a5187 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:25.106045 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0c249-a2d7-43cc-8f73-0f0268606195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:25.108189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0c249-a2d7-43cc-8f73-0f0268606195 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:25.109665 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0c249-a2d7-43cc-8f73-0f0268606195 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:25.110341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0c249-a2d7-43cc-8f73-0f0268606195 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:25.128227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 291/562] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:36:24 2022] PUT /identity/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/d011a40951564323bf7a9c5601df1470/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:36:25.130320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8e0c6-bdd7-408d-b5b9-9b82c40a5187 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:25.308063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0c249-a2d7-43cc-8f73-0f0268606195 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:25.309914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0c249-a2d7-43cc-8f73-0f0268606195 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:25.331981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8e0c6-bdd7-408d-b5b9-9b82c40a5187 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:25.396739 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d480a-232a-4c71-8b83-a922c3b7d3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:25.398624 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d480a-232a-4c71-8b83-a922c3b7d3b1 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:25.399223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d480a-232a-4c71-8b83-a922c3b7d3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:25.399800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d480a-232a-4c71-8b83-a922c3b7d3b1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:25.423500 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d480a-232a-4c71-8b83-a922c3b7d3b1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:25.426906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d480a-232a-4c71-8b83-a922c3b7d3b1 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:25.478834 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-eb3d480a-232a-4c71-8b83-a922c3b7d3b1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:25.481341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:25.483301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:25.493876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:25.493876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:25.493876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:25.493876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:25.493876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:25.493876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:25.493876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:25.493876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:25.493876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 13:36:25.493876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:25.493876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 292/563] 10.222.0.36 () {56 vars in 1117 bytes} [Sat Oct 15 13:36:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:25.497393 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d480a-232a-4c71-8b83-a922c3b7d3b1 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:25.509884 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:25.615426 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 272/564] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:36:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 816 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 13:36:25.625541 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0c249-a2d7-43cc-8f73-0f0268606195 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:25.707795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d480a-232a-4c71-8b83-a922c3b7d3b1 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:25.764103 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1989e-f48e-4890-9dbe-58a491435f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:25.765063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1989e-f48e-4890-9dbe-58a491435f8f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:25.765511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1989e-f48e-4890-9dbe-58a491435f8f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:25.765951 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1989e-f48e-4890-9dbe-58a491435f8f None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:25.767038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1989e-f48e-4890-9dbe-58a491435f8f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:25.767555 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1989e-f48e-4890-9dbe-58a491435f8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:25.769967 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1989e-f48e-4890-9dbe-58a491435f8f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:25.807334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 293/565] 10.222.0.36 () {56 vars in 1149 bytes} [Sat Oct 15 13:36:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:25.807535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0c249-a2d7-43cc-8f73-0f0268606195 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:25.808028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde97671-aa1e-4c1a-bf78-4212d6c0dfbe None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:25.808028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde97671-aa1e-4c1a-bf78-4212d6c0dfbe None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:25.808028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde97671-aa1e-4c1a-bf78-4212d6c0dfbe None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:25.808028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 294/566] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:25.809664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dce11e-181c-46bf-94c0-1c91c0502e11 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:25.810131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dce11e-181c-46bf-94c0-1c91c0502e11 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:25.810605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dce11e-181c-46bf-94c0-1c91c0502e11 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:25.813140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 295/567] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:25.814474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dce11e-181c-46bf-94c0-1c91c0502e11 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:25.868062 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b3812-781d-460c-9c07-d860f17ee0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:25.869376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b3812-781d-460c-9c07-d860f17ee0ae None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:25.869847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b3812-781d-460c-9c07-d860f17ee0ae None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:25.871013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b3812-781d-460c-9c07-d860f17ee0ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:25.888010 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b3812-781d-460c-9c07-d860f17ee0ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:25.889647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b3812-781d-460c-9c07-d860f17ee0ae None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:25.897919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-3c7b3812-781d-460c-9c07-d860f17ee0ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:25.906226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:25.907470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:25.908942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:25.908942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:25.908942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:25.914452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 273/568] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:25.914452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb020f-9b12-485f-943b-710e49d32f32 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:25.914452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb020f-9b12-485f-943b-710e49d32f32 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:25.914452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb020f-9b12-485f-943b-710e49d32f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:25.930879 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-1beb020f-9b12-485f-943b-710e49d32f32 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:25.932300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1beb020f-9b12-485f-943b-710e49d32f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:25.934740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:25.968957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dce11e-181c-46bf-94c0-1c91c0502e11 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:26.015587 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8cd93-0b9d-40d7-8698-c808ad616e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:26.016466 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8cd93-0b9d-40d7-8698-c808ad616e75 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:26.017167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8cd93-0b9d-40d7-8698-c808ad616e75 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:26.017561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8cd93-0b9d-40d7-8698-c808ad616e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:26.020074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 274/569] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:26.021770 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f12ec3-9fde-430d-bf08-16e9204f24b2 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:26.022180 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f12ec3-9fde-430d-bf08-16e9204f24b2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:26.022575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f12ec3-9fde-430d-bf08-16e9204f24b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:26.040081 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8cd93-0b9d-40d7-8698-c808ad616e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:26.042684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8cd93-0b9d-40d7-8698-c808ad616e75 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:26.055920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-22f12ec3-9fde-430d-bf08-16e9204f24b2 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:26.058922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-22f12ec3-9fde-430d-bf08-16e9204f24b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-a2b8cd93-0b9d-40d7-8698-c808ad616e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:26.069277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:26.071175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:26.072949 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:26.072949 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:26.072949 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:26.072949 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 296/570] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:26.073386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8cd93-0b9d-40d7-8698-c808ad616e75 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:26.186852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 275/571] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:26.187645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f12ec3-9fde-430d-bf08-16e9204f24b2 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:26.285954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8cd93-0b9d-40d7-8698-c808ad616e75 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:26.320372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:26.328907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:26.386571 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cfecb-eb2a-4d26-b760-a5fdad079111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:26.387195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cfecb-eb2a-4d26-b760-a5fdad079111 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:26.387475 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cfecb-eb2a-4d26-b760-a5fdad079111 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:26.387743 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cfecb-eb2a-4d26-b760-a5fdad079111 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:26.388443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cfecb-eb2a-4d26-b760-a5fdad079111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:26.388745 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cfecb-eb2a-4d26-b760-a5fdad079111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:26.390147 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cfecb-eb2a-4d26-b760-a5fdad079111 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:26.406937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f12ec3-9fde-430d-bf08-16e9204f24b2 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:26.430330 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 297/572] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:26.434247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cfecb-eb2a-4d26-b760-a5fdad079111 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:26.461505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4f136-ac7b-4590-891a-547a3f88677a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:26.463156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4f136-ac7b-4590-891a-547a3f88677a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:26.463639 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4f136-ac7b-4590-891a-547a3f88677a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:26.464142 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4f136-ac7b-4590-891a-547a3f88677a None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:26.465333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4f136-ac7b-4590-891a-547a3f88677a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:26.465868 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4f136-ac7b-4590-891a-547a3f88677a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:26.469350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4f136-ac7b-4590-891a-547a3f88677a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:26.497546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 276/573] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:26.499313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4f136-ac7b-4590-891a-547a3f88677a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:26.649091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cfecb-eb2a-4d26-b760-a5fdad079111 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:26.709768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4f136-ac7b-4590-891a-547a3f88677a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:26.721028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eb74d-c861-4bd5-8a5f-30cd97c1736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:26.721944 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eb74d-c861-4bd5-8a5f-30cd97c1736c None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:26.722381 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eb74d-c861-4bd5-8a5f-30cd97c1736c None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:26.722793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eb74d-c861-4bd5-8a5f-30cd97c1736c None admin] PATH_INFO: `/v3/services/network` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:26.723544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6eb74d-c861-4bd5-8a5f-30cd97c1736c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:26.725979 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6eb74d-c861-4bd5-8a5f-30cd97c1736c None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-bc6eb74d-c861-4bd5-8a5f-30cd97c1736c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:26.741141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:26.743324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:26.745225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:26.745225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:26.745225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:26.745225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:26.745225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:26.745225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:26.745225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:26.745225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:26.745225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:26.745225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:26.745225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:26.745225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:26.745225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 13:36:26.745225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:26.751763 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 298/574] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:26 2022] GET /identity/v3/services/network => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:26.753129 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eb74d-c861-4bd5-8a5f-30cd97c1736c None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:26.771825 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:26.788490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1ce78-7bdf-4e60-ae03-a5c5b45b941f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:26.789892 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1ce78-7bdf-4e60-ae03-a5c5b45b941f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:26.790365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1ce78-7bdf-4e60-ae03-a5c5b45b941f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:26.791083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1ce78-7bdf-4e60-ae03-a5c5b45b941f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:26.801591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1ce78-7bdf-4e60-ae03-a5c5b45b941f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:26.803084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1ce78-7bdf-4e60-ae03-a5c5b45b941f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:26.806006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-01d1ce78-7bdf-4e60-ae03-a5c5b45b941f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:26.817179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:26.818713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:26.821455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:26.821455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:26.821455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:26.821455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:26.821455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:26.821455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:26.821455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:26.821455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:26.821455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 277/575] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:26.821455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1ce78-7bdf-4e60-ae03-a5c5b45b941f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:26.908229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eb74d-c861-4bd5-8a5f-30cd97c1736c None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:26.960360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1ce78-7bdf-4e60-ae03-a5c5b45b941f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:26.967710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df48c1-6b3b-4938-b8ee-490006a4acf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:26.969777 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df48c1-6b3b-4938-b8ee-490006a4acf1 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:26.970457 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df48c1-6b3b-4938-b8ee-490006a4acf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:26.971168 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df48c1-6b3b-4938-b8ee-490006a4acf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:26.982533 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df48c1-6b3b-4938-b8ee-490006a4acf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:26.985683 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df48c1-6b3b-4938-b8ee-490006a4acf1 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:27.003823 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf3497-20ea-4ad2-814d-931545948b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:27.004502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf3497-20ea-4ad2-814d-931545948b5e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:27.004777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf3497-20ea-4ad2-814d-931545948b5e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:27.005041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf3497-20ea-4ad2-814d-931545948b5e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:27.005708 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf3497-20ea-4ad2-814d-931545948b5e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:27.007105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf3497-20ea-4ad2-814d-931545948b5e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-06df48c1-6b3b-4938-b8ee-490006a4acf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:27.010906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:27.012820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:27.015056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:27.015056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:27.015056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:27.015056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:27.015056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:27.015056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:27.015056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:27.015056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-7bdf3497-20ea-4ad2-814d-931545948b5e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:27.018033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:27.028697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:27.031684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 278/576] 10.222.0.36 () {56 vars in 1129 bytes} [Sat Oct 15 13:36:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:27.034648 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf3497-20ea-4ad2-814d-931545948b5e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:27.034846 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 299/577] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:27.034846 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df48c1-6b3b-4938-b8ee-490006a4acf1 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:27.122063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:27.196865 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:27.206305 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df48c1-6b3b-4938-b8ee-490006a4acf1 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:27.224423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf3497-20ea-4ad2-814d-931545948b5e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:27.250650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c7d92-3198-445d-87cf-fae0a3bd5140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:27.251476 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c7d92-3198-445d-87cf-fae0a3bd5140 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:27.251757 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c7d92-3198-445d-87cf-fae0a3bd5140 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:27.252073 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c7d92-3198-445d-87cf-fae0a3bd5140 None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:27.252816 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c7d92-3198-445d-87cf-fae0a3bd5140 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:27.253129 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c7d92-3198-445d-87cf-fae0a3bd5140 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:27.254673 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c7d92-3198-445d-87cf-fae0a3bd5140 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:27.272215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 300/578] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:27.273121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c7d92-3198-445d-87cf-fae0a3bd5140 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:27.299779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0003a09-7371-428d-8fc5-a91f32f512fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:27.300802 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0003a09-7371-428d-8fc5-a91f32f512fb None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:27.301789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0003a09-7371-428d-8fc5-a91f32f512fb None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:27.302259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0003a09-7371-428d-8fc5-a91f32f512fb None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:27.303325 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0003a09-7371-428d-8fc5-a91f32f512fb None admin] RBAC: Adding query filter params (name=network) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:27.303910 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0003a09-7371-428d-8fc5-a91f32f512fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:27.305601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0003a09-7371-428d-8fc5-a91f32f512fb None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:27.317658 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 279/579] 10.222.0.36 () {56 vars in 1112 bytes} [Sat Oct 15 13:36:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:27.318945 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0003a09-7371-428d-8fc5-a91f32f512fb None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:27.474120 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c7d92-3198-445d-87cf-fae0a3bd5140 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:27.493462 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0003a09-7371-428d-8fc5-a91f32f512fb None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:27.522474 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:27.546118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24868b-a0fe-4c7d-b3be-dbdda5591097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:27.547314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24868b-a0fe-4c7d-b3be-dbdda5591097 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:27.548058 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24868b-a0fe-4c7d-b3be-dbdda5591097 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:27.549347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24868b-a0fe-4c7d-b3be-dbdda5591097 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:27.550446 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24868b-a0fe-4c7d-b3be-dbdda5591097 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:27.550951 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24868b-a0fe-4c7d-b3be-dbdda5591097 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:27.553398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24868b-a0fe-4c7d-b3be-dbdda5591097 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:27.554308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44260e-9897-4c80-9e76-b4f78d3b6d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:27.555246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44260e-9897-4c80-9e76-b4f78d3b6d00 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:27.555693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44260e-9897-4c80-9e76-b4f78d3b6d00 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:27.556183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44260e-9897-4c80-9e76-b4f78d3b6d00 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:27.557325 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44260e-9897-4c80-9e76-b4f78d3b6d00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:27.557823 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44260e-9897-4c80-9e76-b4f78d3b6d00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:27.560349 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44260e-9897-4c80-9e76-b4f78d3b6d00 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:27.563212 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 301/580] 10.222.0.36 () {56 vars in 1124 bytes} [Sat Oct 15 13:36:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:27.565736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24868b-a0fe-4c7d-b3be-dbdda5591097 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:27.583008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 280/581] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:27.584382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44260e-9897-4c80-9e76-b4f78d3b6d00 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:27.600316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:27.696763 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24868b-a0fe-4c7d-b3be-dbdda5591097 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:27.713623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44260e-9897-4c80-9e76-b4f78d3b6d00 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:27.738474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aaf3c4-aa11-4c85-8085-1c7fc95322d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:27.739101 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aaf3c4-aa11-4c85-8085-1c7fc95322d8 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:27.739382 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aaf3c4-aa11-4c85-8085-1c7fc95322d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:27.739636 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aaf3c4-aa11-4c85-8085-1c7fc95322d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:27.740386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aaf3c4-aa11-4c85-8085-1c7fc95322d8 None admin] RBAC: Adding query filter params (role.id=57740dc2ebe1429294f20dd51592b82a, scope.project.id=2a5a64cc8aa04972bb464407cb0312b4, user.id=2e13692c1e83413a909d99bb7db2e4b8) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:27.740682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aaf3c4-aa11-4c85-8085-1c7fc95322d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:27.742120 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aaf3c4-aa11-4c85-8085-1c7fc95322d8 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:27.749035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 302/582] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:36:27 2022] GET /identity/v3/role_assignments?user.id=2e13692c1e83413a909d99bb7db2e4b8&scope.project.id=2a5a64cc8aa04972bb464407cb0312b4&role.id=57740dc2ebe1429294f20dd51592b82a => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:27.749828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aaf3c4-aa11-4c85-8085-1c7fc95322d8 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:27.763376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ffb73-4ec5-477a-8213-f09ea6d147cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:27.764820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ffb73-4ec5-477a-8213-f09ea6d147cc None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:27.765549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ffb73-4ec5-477a-8213-f09ea6d147cc None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:27.766026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ffb73-4ec5-477a-8213-f09ea6d147cc None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:27.766914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ffb73-4ec5-477a-8213-f09ea6d147cc None admin] RBAC: Adding query filter params (type=network) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:27.767569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ffb73-4ec5-477a-8213-f09ea6d147cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:27.770049 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ffb73-4ec5-477a-8213-f09ea6d147cc None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:27.778964 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 281/583] 10.222.0.36 () {56 vars in 1112 bytes} [Sat Oct 15 13:36:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:27.780734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ffb73-4ec5-477a-8213-f09ea6d147cc None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:27.841672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:27.867388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:27.893042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aaf3c4-aa11-4c85-8085-1c7fc95322d8 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:27.942200 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb2ce1-9773-4d8f-9b8e-9ad7703de77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:27.944037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb2ce1-9773-4d8f-9b8e-9ad7703de77f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:27.945026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb2ce1-9773-4d8f-9b8e-9ad7703de77f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:27.946665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb2ce1-9773-4d8f-9b8e-9ad7703de77f None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:27.948916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb2ce1-9773-4d8f-9b8e-9ad7703de77f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:27.951223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb2ce1-9773-4d8f-9b8e-9ad7703de77f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:27.952752 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ffb73-4ec5-477a-8213-f09ea6d147cc None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:27.955763 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb2ce1-9773-4d8f-9b8e-9ad7703de77f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:27.973619 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 303/584] 10.222.0.36 () {56 vars in 1124 bytes} [Sat Oct 15 13:36:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:27.975522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb2ce1-9773-4d8f-9b8e-9ad7703de77f None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:28.004795 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e37a82-b514-4cb0-8cee-1ae36db1360e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:28.005327 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e37a82-b514-4cb0-8cee-1ae36db1360e None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:28.005704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e37a82-b514-4cb0-8cee-1ae36db1360e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:28.006068 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e37a82-b514-4cb0-8cee-1ae36db1360e None admin] PATH_INFO: `/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/2f06388453f44fca9e30fbc37da90bba/roles/8b837ae2ed57479a93acb638b80e2d67` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:28.025300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e37a82-b514-4cb0-8cee-1ae36db1360e None admin] RBAC: Authorizing `identity:create_grant(project_id=2a5a64cc8aa04972bb464407cb0312b4, user_id=2f06388453f44fca9e30fbc37da90bba, role_id=8b837ae2ed57479a93acb638b80e2d67)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:28.026937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e37a82-b514-4cb0-8cee-1ae36db1360e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:28.049638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 282/585] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:36:27 2022] PUT /identity/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/2f06388453f44fca9e30fbc37da90bba/roles/8b837ae2ed57479a93acb638b80e2d67 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:36:28.050829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e37a82-b514-4cb0-8cee-1ae36db1360e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:28.162433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb2ce1-9773-4d8f-9b8e-9ad7703de77f None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:28.235307 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3642e-e1a6-4742-83a4-f2fc5be5456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:28.236338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:28.236838 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3642e-e1a6-4742-83a4-f2fc5be5456b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:28.237176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3642e-e1a6-4742-83a4-f2fc5be5456b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:28.237446 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3642e-e1a6-4742-83a4-f2fc5be5456b None admin] PATH_INFO: `/v3/endpoints` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:28.238626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3642e-e1a6-4742-83a4-f2fc5be5456b None admin] RBAC: Adding query filter params (interface=public, service_id=0b281acd0d074a6db921f647f8d1e30d, region_id=RegionOne) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:28.238626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3642e-e1a6-4742-83a4-f2fc5be5456b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:28.241697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3642e-e1a6-4742-83a4-f2fc5be5456b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:28.260929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e37a82-b514-4cb0-8cee-1ae36db1360e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:28.266425 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 304/586] 10.222.0.36 () {56 vars in 1251 bytes} [Sat Oct 15 13:36:27 2022] GET /identity/v3/endpoints?service_id=0b281acd0d074a6db921f647f8d1e30d&interface=public®ion_id=RegionOne => generated 194 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:28.269386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fab33-9bdb-4467-80be-b1ea2234be79 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:28.270311 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fab33-9bdb-4467-80be-b1ea2234be79 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:28.272100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fab33-9bdb-4467-80be-b1ea2234be79 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:28.276631 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 305/587] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:28.278129 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fab33-9bdb-4467-80be-b1ea2234be79 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:28.303072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:28.330911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120db23-a8e1-42bf-b885-3abd4697543a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:28.332539 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120db23-a8e1-42bf-b885-3abd4697543a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:28.333102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120db23-a8e1-42bf-b885-3abd4697543a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:28.333580 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120db23-a8e1-42bf-b885-3abd4697543a None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:28.334961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8120db23-a8e1-42bf-b885-3abd4697543a None admin] RBAC: Adding query filter params (interface=public, service_id=cab912a78eb04b51a332f4b0e1bcfc95, region_id=RegionOne) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:28.335745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8120db23-a8e1-42bf-b885-3abd4697543a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:28.338733 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8120db23-a8e1-42bf-b885-3abd4697543a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:28.355804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 283/588] 10.222.0.36 () {56 vars in 1251 bytes} [Sat Oct 15 13:36:28 2022] GET /identity/v3/endpoints?service_id=cab912a78eb04b51a332f4b0e1bcfc95&interface=public®ion_id=RegionOne => generated 194 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:28.357445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1648f8-faaf-48c3-8cc4-70e756bce430 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:28.357898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1648f8-faaf-48c3-8cc4-70e756bce430 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:28.358363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1648f8-faaf-48c3-8cc4-70e756bce430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:28.389236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-da1648f8-faaf-48c3-8cc4-70e756bce430 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:28.391395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-da1648f8-faaf-48c3-8cc4-70e756bce430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:28.418047 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fab33-9bdb-4467-80be-b1ea2234be79 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:28.459086 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7abcc-91fc-416b-ad4e-aeb35abe84cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:28.459706 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7abcc-91fc-416b-ad4e-aeb35abe84cd None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:28.460011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7abcc-91fc-416b-ad4e-aeb35abe84cd None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:28.460692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7abcc-91fc-416b-ad4e-aeb35abe84cd None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:28.460951 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7abcc-91fc-416b-ad4e-aeb35abe84cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:28.462531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7abcc-91fc-416b-ad4e-aeb35abe84cd None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:28.469158 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 306/589] 10.222.0.36 () {56 vars in 1087 bytes} [Sat Oct 15 13:36:28 2022] GET /identity/v3/services => generated 772 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:28.470149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7abcc-91fc-416b-ad4e-aeb35abe84cd None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:28.494701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 284/590] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:28.562430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:28.592819 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7abcc-91fc-416b-ad4e-aeb35abe84cd None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:28.633345 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c07f5d-b88b-4c6e-a062-f2f1437b9161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:28.634367 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c07f5d-b88b-4c6e-a062-f2f1437b9161 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:28.634839 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c07f5d-b88b-4c6e-a062-f2f1437b9161 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:28.635301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c07f5d-b88b-4c6e-a062-f2f1437b9161 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:28.636251 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c07f5d-b88b-4c6e-a062-f2f1437b9161 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:28.637987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c07f5d-b88b-4c6e-a062-f2f1437b9161 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:28.645077 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 307/591] 10.222.0.36 () {56 vars in 1087 bytes} [Sat Oct 15 13:36:28 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:28.809500 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1648f8-faaf-48c3-8cc4-70e756bce430 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:28.930013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:28.952133 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1648f8-faaf-48c3-8cc4-70e756bce430 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:28.997011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3eb52-d0be-4279-af91-77508bb64a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:28.997668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3eb52-d0be-4279-af91-77508bb64a4a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:28.997940 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3eb52-d0be-4279-af91-77508bb64a4a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:28.998206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3eb52-d0be-4279-af91-77508bb64a4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:29.014863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3eb52-d0be-4279-af91-77508bb64a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-ddd3eb52-d0be-4279-af91-77508bb64a4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:36:29.019929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:29.020920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:29.021936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:29.021936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:29.021936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:29.021936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:29.021936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:36:29.021936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:36:29.021936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:36:29.021936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:36:29.021936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:36:29.021936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:36:29.021936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:36:29.021936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:29.021936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 285/592] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:36:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:36:29.029365 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c07f5d-b88b-4c6e-a062-f2f1437b9161 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:29.143608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c07f5d-b88b-4c6e-a062-f2f1437b9161 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:29.187732 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6f561-a201-4605-9341-4b30d8b0480f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:29.188489 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6f561-a201-4605-9341-4b30d8b0480f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:29.188885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6f561-a201-4605-9341-4b30d8b0480f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:29.189257 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6f561-a201-4605-9341-4b30d8b0480f None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:29.189957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6f561-a201-4605-9341-4b30d8b0480f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:29.190259 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6f561-a201-4605-9341-4b30d8b0480f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:29.191670 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6f561-a201-4605-9341-4b30d8b0480f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:29.201394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 308/593] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:36:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:36:29.210669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3eb52-d0be-4279-af91-77508bb64a4a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:29.352408 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3eb52-d0be-4279-af91-77508bb64a4a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:29.400541 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f45e0-08f8-4076-b6a2-d58983a98d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:29.401110 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f45e0-08f8-4076-b6a2-d58983a98d23 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:29.401379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f45e0-08f8-4076-b6a2-d58983a98d23 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:29.401644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f45e0-08f8-4076-b6a2-d58983a98d23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:29.418345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f45e0-08f8-4076-b6a2-d58983a98d23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:29.419850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f45e0-08f8-4076-b6a2-d58983a98d23 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:29.424286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-3b4f45e0-08f8-4076-b6a2-d58983a98d23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:29.445958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:29.447986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:29.450270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:29.450270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:29.450270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:29.450270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 286/594] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:29.455777 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6f561-a201-4605-9341-4b30d8b0480f None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:29.458784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:29.575197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6f561-a201-4605-9341-4b30d8b0480f None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:29.617496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56037301-1f57-4bd9-8f04-8abb46af1e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:29.618142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56037301-1f57-4bd9-8f04-8abb46af1e5e None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:29.618416 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56037301-1f57-4bd9-8f04-8abb46af1e5e None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:29.618691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56037301-1f57-4bd9-8f04-8abb46af1e5e None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:29.619361 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56037301-1f57-4bd9-8f04-8abb46af1e5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:29.619661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56037301-1f57-4bd9-8f04-8abb46af1e5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:29.621136 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56037301-1f57-4bd9-8f04-8abb46af1e5e None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:29.640384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 309/595] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:29.648895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f45e0-08f8-4076-b6a2-d58983a98d23 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:29.786638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f45e0-08f8-4076-b6a2-d58983a98d23 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:29.840317 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ebf37-c123-46a9-a1e5-2b4c39d32d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:29.840971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ebf37-c123-46a9-a1e5-2b4c39d32d80 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:29.841239 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ebf37-c123-46a9-a1e5-2b4c39d32d80 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:29.841499 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ebf37-c123-46a9-a1e5-2b4c39d32d80 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:29.856662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ebf37-c123-46a9-a1e5-2b4c39d32d80 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:29.858126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ebf37-c123-46a9-a1e5-2b4c39d32d80 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-7c8ebf37-c123-46a9-a1e5-2b4c39d32d80 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:29.876693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:29.878947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:29.881221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:29.881221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:29.881221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:29.881221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:29.881221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:29.881221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:29.881221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:29.881221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:29.881221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 15 13:36:29.881221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:29.881221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 287/596] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:36:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:29.890915 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56037301-1f57-4bd9-8f04-8abb46af1e5e None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:30.038386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:30.138285 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56037301-1f57-4bd9-8f04-8abb46af1e5e None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:30.216734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a50e4-40e7-4e24-b977-33c947a25a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:30.217789 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a50e4-40e7-4e24-b977-33c947a25a69 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:30.218281 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a50e4-40e7-4e24-b977-33c947a25a69 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:30.218762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a50e4-40e7-4e24-b977-33c947a25a69 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:30.220663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a50e4-40e7-4e24-b977-33c947a25a69 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:30.221188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a50e4-40e7-4e24-b977-33c947a25a69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:30.223947 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a50e4-40e7-4e24-b977-33c947a25a69 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:30.270671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 310/597] 10.222.0.36 () {56 vars in 1135 bytes} [Sat Oct 15 13:36:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:36:30.280564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ebf37-c123-46a9-a1e5-2b4c39d32d80 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:30.410030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:30.494073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ebf37-c123-46a9-a1e5-2b4c39d32d80 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:30.545615 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767b3cc-55c0-4369-be01-73ec663e43e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:30.546629 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767b3cc-55c0-4369-be01-73ec663e43e5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:30.547106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767b3cc-55c0-4369-be01-73ec663e43e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:30.547576 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767b3cc-55c0-4369-be01-73ec663e43e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:30.635753 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0767b3cc-55c0-4369-be01-73ec663e43e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:30.637255 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0767b3cc-55c0-4369-be01-73ec663e43e5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:30.697207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:30.746333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 288/598] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:36:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 13:36:30.755549 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a50e4-40e7-4e24-b977-33c947a25a69 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:30.942796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a50e4-40e7-4e24-b977-33c947a25a69 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:30.973632 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:31.006538 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81cfb2-185e-46cd-a60e-39e1232a9e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:31.007596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81cfb2-185e-46cd-a60e-39e1232a9e12 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:31.008131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81cfb2-185e-46cd-a60e-39e1232a9e12 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:31.008571 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81cfb2-185e-46cd-a60e-39e1232a9e12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:31.034653 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81cfb2-185e-46cd-a60e-39e1232a9e12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:31.037380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81cfb2-185e-46cd-a60e-39e1232a9e12 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-4e81cfb2-185e-46cd-a60e-39e1232a9e12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:31.074724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:31.075804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:31.076741 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:31.076741 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:31.076741 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:31.077362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 311/599] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:31.087196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767b3cc-55c0-4369-be01-73ec663e43e5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:31.264827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767b3cc-55c0-4369-be01-73ec663e43e5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:31.327555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae74d4-f35a-4208-b67e-ce18ff7f320a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:31.328256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae74d4-f35a-4208-b67e-ce18ff7f320a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:31.328507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae74d4-f35a-4208-b67e-ce18ff7f320a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:31.328722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae74d4-f35a-4208-b67e-ce18ff7f320a None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:31.329574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae74d4-f35a-4208-b67e-ce18ff7f320a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:31.329870 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae74d4-f35a-4208-b67e-ce18ff7f320a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:31.331266 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae74d4-f35a-4208-b67e-ce18ff7f320a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:31.350379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 289/600] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:31.360241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81cfb2-185e-46cd-a60e-39e1232a9e12 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:31.630369 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81cfb2-185e-46cd-a60e-39e1232a9e12 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:31.714812 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617f077-061f-4b84-a633-8c8c214ed8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:31.716020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617f077-061f-4b84-a633-8c8c214ed8fc None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:31.716523 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617f077-061f-4b84-a633-8c8c214ed8fc None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:31.717017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617f077-061f-4b84-a633-8c8c214ed8fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:31.742119 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617f077-061f-4b84-a633-8c8c214ed8fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:31.745139 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617f077-061f-4b84-a633-8c8c214ed8fc None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-3617f077-061f-4b84-a633-8c8c214ed8fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:31.778751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:31.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:31.781116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:31.781116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:31.781116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:31.781116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:31.781116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:31.781116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:31.781116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:31.781116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:31.781568 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 312/601] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:31.793576 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae74d4-f35a-4208-b67e-ce18ff7f320a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:31.822161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a09b4e-3562-4ec7-a7f9-e6f36b802fcc None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:31.822627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a09b4e-3562-4ec7-a7f9-e6f36b802fcc None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:31.823091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a09b4e-3562-4ec7-a7f9-e6f36b802fcc None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:31.826037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 313/602] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:31.828896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:31.842840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c33603-9e07-4e3b-b8ff-a82362684daf None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:31.843311 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c33603-9e07-4e3b-b8ff-a82362684daf None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:31.843780 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c33603-9e07-4e3b-b8ff-a82362684daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:31.873178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-84c33603-9e07-4e3b-b8ff-a82362684daf None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:31.874571 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-84c33603-9e07-4e3b-b8ff-a82362684daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:31.912980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:31.947408 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae74d4-f35a-4208-b67e-ce18ff7f320a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:31.979916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 314/603] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:36:31.979916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86cd52-a6b2-4a3a-aa65-d512a680f610 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:31.979916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86cd52-a6b2-4a3a-aa65-d512a680f610 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:31.979916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86cd52-a6b2-4a3a-aa65-d512a680f610 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:31.979916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 315/604] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:31.979916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dec49-2146-4872-b495-5aa81678c17d None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:31.979916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dec49-2146-4872-b495-5aa81678c17d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:31.980863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dec49-2146-4872-b495-5aa81678c17d None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:31.981520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 316/605] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:31.990082 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadac891-6930-4dd9-abc7-cd9204aa57c8 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:31.990381 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadac891-6930-4dd9-abc7-cd9204aa57c8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:31.990715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadac891-6930-4dd9-abc7-cd9204aa57c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:32.011259 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-eadac891-6930-4dd9-abc7-cd9204aa57c8 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:32.012540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-eadac891-6930-4dd9-abc7-cd9204aa57c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:32.016857 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8dd27d-d10a-4663-8117-499b5114c960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:32.018350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dd27d-d10a-4663-8117-499b5114c960 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:32.019064 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dd27d-d10a-4663-8117-499b5114c960 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:32.019968 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dd27d-d10a-4663-8117-499b5114c960 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:32.021682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8dd27d-d10a-4663-8117-499b5114c960 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:32.022702 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8dd27d-d10a-4663-8117-499b5114c960 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:32.025809 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8dd27d-d10a-4663-8117-499b5114c960 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:32.052344 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 290/606] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:32.054340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53203b9f-cc7e-4b45-be8b-0596bb106ae6 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:32.054893 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53203b9f-cc7e-4b45-be8b-0596bb106ae6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:32.055535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53203b9f-cc7e-4b45-be8b-0596bb106ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:32.079560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-53203b9f-cc7e-4b45-be8b-0596bb106ae6 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:32.080881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-53203b9f-cc7e-4b45-be8b-0596bb106ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:32.103287 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 317/607] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:32.104479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadac891-6930-4dd9-abc7-cd9204aa57c8 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:32.168568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 291/608] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:32.170296 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276ea84-4dc6-4fa9-b2ee-bd995a4d4d3b None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:32.170749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276ea84-4dc6-4fa9-b2ee-bd995a4d4d3b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:32.171203 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276ea84-4dc6-4fa9-b2ee-bd995a4d4d3b None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:32.172949 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 292/609] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:32.180315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee44d9-8171-4423-a03f-b8b4a1663563 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:32.185014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee44d9-8171-4423-a03f-b8b4a1663563 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:32.185501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee44d9-8171-4423-a03f-b8b4a1663563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:32.219635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:32.238643 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e9ee44d9-8171-4423-a03f-b8b4a1663563 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:32.243269 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e9ee44d9-8171-4423-a03f-b8b4a1663563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:32.309183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:32.321289 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadac891-6930-4dd9-abc7-cd9204aa57c8 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:32.367912 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e634b-f519-4961-8932-e444578b46be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:32.368836 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e634b-f519-4961-8932-e444578b46be None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:32.369119 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e634b-f519-4961-8932-e444578b46be None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:32.369388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e634b-f519-4961-8932-e444578b46be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:32.370538 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e634b-f519-4961-8932-e444578b46be None admin] RBAC: Adding query filter params (role.id=57740dc2ebe1429294f20dd51592b82a, scope.project.id=2a5a64cc8aa04972bb464407cb0312b4, user.id=d011a40951564323bf7a9c5601df1470) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:32.370848 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e634b-f519-4961-8932-e444578b46be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:32.372697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e634b-f519-4961-8932-e444578b46be None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:32.380851 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 318/610] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:36:32 2022] GET /identity/v3/role_assignments?user.id=d011a40951564323bf7a9c5601df1470&scope.project.id=2a5a64cc8aa04972bb464407cb0312b4&role.id=57740dc2ebe1429294f20dd51592b82a => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:32.381970 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e634b-f519-4961-8932-e444578b46be None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:32.391325 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 293/611] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:32.394277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee44d9-8171-4423-a03f-b8b4a1663563 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:32.534420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee44d9-8171-4423-a03f-b8b4a1663563 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:32.575122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af64770-c42a-45bc-aa26-a1586c449b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:32.575799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af64770-c42a-45bc-aa26-a1586c449b61 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:32.576139 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af64770-c42a-45bc-aa26-a1586c449b61 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:32.576419 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af64770-c42a-45bc-aa26-a1586c449b61 None admin] PATH_INFO: `/v3/services/network` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:32.577031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af64770-c42a-45bc-aa26-a1586c449b61 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:32.578504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af64770-c42a-45bc-aa26-a1586c449b61 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:32.582306 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-8af64770-c42a-45bc-aa26-a1586c449b61 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:32.590328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:32.591867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:32.593302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 294/612] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:32 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:32.594381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af64770-c42a-45bc-aa26-a1586c449b61 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:32.614816 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e634b-f519-4961-8932-e444578b46be None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:32.681382 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dac82-125e-42a4-9125-6a93312dfc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:32.682379 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dac82-125e-42a4-9125-6a93312dfc60 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:32.682849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dac82-125e-42a4-9125-6a93312dfc60 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:32.683292 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dac82-125e-42a4-9125-6a93312dfc60 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:32.684911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dac82-125e-42a4-9125-6a93312dfc60 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:32.686533 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dac82-125e-42a4-9125-6a93312dfc60 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-556dac82-125e-42a4-9125-6a93312dfc60 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:32.703493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:32.704665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:32.705784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 319/613] 10.222.0.36 () {56 vars in 1132 bytes} [Sat Oct 15 13:36:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:32.708358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dac82-125e-42a4-9125-6a93312dfc60 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:32.726817 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af64770-c42a-45bc-aa26-a1586c449b61 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:32.747533 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:32.780914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c0ba2-b043-44ca-b4f2-0704367c870b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:32.781872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c0ba2-b043-44ca-b4f2-0704367c870b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:32.782296 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c0ba2-b043-44ca-b4f2-0704367c870b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:32.782758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c0ba2-b043-44ca-b4f2-0704367c870b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:32.783678 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c0ba2-b043-44ca-b4f2-0704367c870b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:32.785588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c0ba2-b043-44ca-b4f2-0704367c870b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-b30c0ba2-b043-44ca-b4f2-0704367c870b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:32.795356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:32.797305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:32.798881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 295/614] 10.222.0.36 () {56 vars in 1129 bytes} [Sat Oct 15 13:36:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:32.800330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c0ba2-b043-44ca-b4f2-0704367c870b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:32.890744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:32.898702 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dac82-125e-42a4-9125-6a93312dfc60 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:32.942616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8388d-03e5-47bf-8077-2499d2caf9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:32.943255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8388d-03e5-47bf-8077-2499d2caf9eb None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:32.943530 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8388d-03e5-47bf-8077-2499d2caf9eb None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:32.943828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8388d-03e5-47bf-8077-2499d2caf9eb None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:32.944802 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8388d-03e5-47bf-8077-2499d2caf9eb None admin] RBAC: Adding query filter params (name=network) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:32.945110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8388d-03e5-47bf-8077-2499d2caf9eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:32.946715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8388d-03e5-47bf-8077-2499d2caf9eb None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:32.952310 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 320/615] 10.222.0.36 () {56 vars in 1112 bytes} [Sat Oct 15 13:36:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:32.953169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8388d-03e5-47bf-8077-2499d2caf9eb None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:32.970150 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c0ba2-b043-44ca-b4f2-0704367c870b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:33.024422 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582b4ce-42f9-4a14-9503-6a5239005a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:33.025071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582b4ce-42f9-4a14-9503-6a5239005a69 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:33.025341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582b4ce-42f9-4a14-9503-6a5239005a69 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:33.026094 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582b4ce-42f9-4a14-9503-6a5239005a69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:33.040088 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582b4ce-42f9-4a14-9503-6a5239005a69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-8582b4ce-42f9-4a14-9503-6a5239005a69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:36:33.045864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:33.047696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:33.049810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:33.049810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:33.049810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:33.049810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:33.049810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:36:33.049810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:36:33.049810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:36:33.049810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:36:33.049810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:36:33.049810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:36:33.049810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:36:33.049810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:33.055503 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 296/616] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:36:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:33.056513 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582b4ce-42f9-4a14-9503-6a5239005a69 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:33.062584 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:33.080871 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8388d-03e5-47bf-8077-2499d2caf9eb None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:33.117368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8958749-fddf-4f71-9ef6-8cd7af5c7b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:33.117990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8958749-fddf-4f71-9ef6-8cd7af5c7b17 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:33.118266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8958749-fddf-4f71-9ef6-8cd7af5c7b17 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:33.118597 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8958749-fddf-4f71-9ef6-8cd7af5c7b17 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:33.119242 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8958749-fddf-4f71-9ef6-8cd7af5c7b17 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:33.119575 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8958749-fddf-4f71-9ef6-8cd7af5c7b17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:33.121299 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8958749-fddf-4f71-9ef6-8cd7af5c7b17 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:33.130382 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 321/617] 10.222.0.36 () {56 vars in 1126 bytes} [Sat Oct 15 13:36:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:33.130382 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8958749-fddf-4f71-9ef6-8cd7af5c7b17 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:33.176585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582b4ce-42f9-4a14-9503-6a5239005a69 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:33.202717 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:33.218085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b65fb-c499-4a91-9c7d-ab4736c36c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:33.218840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b65fb-c499-4a91-9c7d-ab4736c36c4e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:33.219183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b65fb-c499-4a91-9c7d-ab4736c36c4e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:33.219509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b65fb-c499-4a91-9c7d-ab4736c36c4e None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:33.220576 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b65fb-c499-4a91-9c7d-ab4736c36c4e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:33.220955 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b65fb-c499-4a91-9c7d-ab4736c36c4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:33.222455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b65fb-c499-4a91-9c7d-ab4736c36c4e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:33.228070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 297/618] 10.222.0.36 () {56 vars in 1124 bytes} [Sat Oct 15 13:36:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:33.233474 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b65fb-c499-4a91-9c7d-ab4736c36c4e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:33.243293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8958749-fddf-4f71-9ef6-8cd7af5c7b17 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:33.280281 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d17923-1198-4f98-aa3d-e52e34c21aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:33.281010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17923-1198-4f98-aa3d-e52e34c21aeb None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:33.281292 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17923-1198-4f98-aa3d-e52e34c21aeb None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:33.281569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17923-1198-4f98-aa3d-e52e34c21aeb None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:33.282358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d17923-1198-4f98-aa3d-e52e34c21aeb None admin] RBAC: Adding query filter params (type=network) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:33.282806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d17923-1198-4f98-aa3d-e52e34c21aeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:33.284452 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d17923-1198-4f98-aa3d-e52e34c21aeb None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:33.296793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 322/619] 10.222.0.36 () {56 vars in 1112 bytes} [Sat Oct 15 13:36:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:33.296793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d17923-1198-4f98-aa3d-e52e34c21aeb None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:33.313832 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:33.358006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b65fb-c499-4a91-9c7d-ab4736c36c4e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:33.400948 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d889be-92a5-476b-b196-9036f04c2154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:33.401714 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d889be-92a5-476b-b196-9036f04c2154 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:33.401923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d889be-92a5-476b-b196-9036f04c2154 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:33.402106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d889be-92a5-476b-b196-9036f04c2154 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:33.402762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d889be-92a5-476b-b196-9036f04c2154 None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:33.403063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d889be-92a5-476b-b196-9036f04c2154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:33.404552 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d889be-92a5-476b-b196-9036f04c2154 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:33.416908 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 298/620] 10.222.0.36 () {56 vars in 1103 bytes} [Sat Oct 15 13:36:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:33.418375 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d889be-92a5-476b-b196-9036f04c2154 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:33.429781 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d17923-1198-4f98-aa3d-e52e34c21aeb None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:33.475332 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6e7b4-d496-4d41-bf87-b7c87532958a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:33.476022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6e7b4-d496-4d41-bf87-b7c87532958a None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:33.476312 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6e7b4-d496-4d41-bf87-b7c87532958a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:33.476559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6e7b4-d496-4d41-bf87-b7c87532958a None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:33.477192 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6e7b4-d496-4d41-bf87-b7c87532958a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:33.477486 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6e7b4-d496-4d41-bf87-b7c87532958a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:33.478885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6e7b4-d496-4d41-bf87-b7c87532958a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:33.491942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 323/621] 10.222.0.36 () {56 vars in 1126 bytes} [Sat Oct 15 13:36:33 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:33.491942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6e7b4-d496-4d41-bf87-b7c87532958a None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:33.514227 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:33.604145 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:33.634425 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d889be-92a5-476b-b196-9036f04c2154 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:33.643931 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6e7b4-d496-4d41-bf87-b7c87532958a None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:33.683969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3822c4-bff1-4e6d-87a8-493101f2d8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:33.684767 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3822c4-bff1-4e6d-87a8-493101f2d8fc None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:33.685096 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3822c4-bff1-4e6d-87a8-493101f2d8fc None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:33.685437 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3822c4-bff1-4e6d-87a8-493101f2d8fc None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:33.686169 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3822c4-bff1-4e6d-87a8-493101f2d8fc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:33.686549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3822c4-bff1-4e6d-87a8-493101f2d8fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:33.688114 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3822c4-bff1-4e6d-87a8-493101f2d8fc None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:33.697441 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 299/622] 10.222.0.36 () {56 vars in 1124 bytes} [Sat Oct 15 13:36:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:33.698776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3822c4-bff1-4e6d-87a8-493101f2d8fc None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:33.717817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de71001-c255-4377-be10-8b4570b0768e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:33.719366 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de71001-c255-4377-be10-8b4570b0768e None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:33.720313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de71001-c255-4377-be10-8b4570b0768e None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:33.720799 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de71001-c255-4377-be10-8b4570b0768e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:33.722063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de71001-c255-4377-be10-8b4570b0768e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:33.726281 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de71001-c255-4377-be10-8b4570b0768e None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:33.772951 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 324/623] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:36:33 2022] POST /identity/v3/endpoints => generated 337 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:36:33.780378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de71001-c255-4377-be10-8b4570b0768e None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:33.819307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3822c4-bff1-4e6d-87a8-493101f2d8fc None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:33.854536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:33.884943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da2f30-1293-4439-a621-d768e3ee965e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:33.885991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da2f30-1293-4439-a621-d768e3ee965e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:33.886510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da2f30-1293-4439-a621-d768e3ee965e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:33.887630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da2f30-1293-4439-a621-d768e3ee965e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:33.911367 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da2f30-1293-4439-a621-d768e3ee965e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:33.912923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da2f30-1293-4439-a621-d768e3ee965e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-91da2f30-1293-4439-a621-d768e3ee965e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:33.948060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:33.949101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:33.949979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:33.949979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:33.949979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:33.952182 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 300/624] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:33.962497 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da2f30-1293-4439-a621-d768e3ee965e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:33.984935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de71001-c255-4377-be10-8b4570b0768e None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:34.059003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:34.071320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080b6a-d69c-49a0-a4dc-6d3f6bf7bf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:34.072425 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080b6a-d69c-49a0-a4dc-6d3f6bf7bf2d None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:34.073911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080b6a-d69c-49a0-a4dc-6d3f6bf7bf2d None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:34.074460 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080b6a-d69c-49a0-a4dc-6d3f6bf7bf2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:34.077126 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5080b6a-d69c-49a0-a4dc-6d3f6bf7bf2d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:34.079116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5080b6a-d69c-49a0-a4dc-6d3f6bf7bf2d None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:34.111353 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 325/625] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:36:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:36:34.220520 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da2f30-1293-4439-a621-d768e3ee965e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:34.305873 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ddc91-b6f9-4d00-b6ea-f83ef0c9840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:34.306961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ddc91-b6f9-4d00-b6ea-f83ef0c9840e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:34.307449 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ddc91-b6f9-4d00-b6ea-f83ef0c9840e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:34.308008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ddc91-b6f9-4d00-b6ea-f83ef0c9840e None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:34.309185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ddc91-b6f9-4d00-b6ea-f83ef0c9840e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:34.309705 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ddc91-b6f9-4d00-b6ea-f83ef0c9840e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:34.312395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ddc91-b6f9-4d00-b6ea-f83ef0c9840e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:34.338063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 301/626] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:34.347225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080b6a-d69c-49a0-a4dc-6d3f6bf7bf2d None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:34.535353 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080b6a-d69c-49a0-a4dc-6d3f6bf7bf2d None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:34.594983 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44159c3d-bcfb-4966-af73-fd5d5c9cf71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:34.596604 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44159c3d-bcfb-4966-af73-fd5d5c9cf71c None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:34.597388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44159c3d-bcfb-4966-af73-fd5d5c9cf71c None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:34.598146 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44159c3d-bcfb-4966-af73-fd5d5c9cf71c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:34.609591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44159c3d-bcfb-4966-af73-fd5d5c9cf71c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:34.611566 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44159c3d-bcfb-4966-af73-fd5d5c9cf71c None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-44159c3d-bcfb-4966-af73-fd5d5c9cf71c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:34.624302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:34.626581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:34.629341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:34.629341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:34.629341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:34.629341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:34.629341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:34.629341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:34.629341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:34.629341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:34.629341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 13:36:34.629341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:34.635100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 326/627] 10.222.0.36 () {56 vars in 1117 bytes} [Sat Oct 15 13:36:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:34.635511 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ddc91-b6f9-4d00-b6ea-f83ef0c9840e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:34.657488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:34.750355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ddc91-b6f9-4d00-b6ea-f83ef0c9840e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:34.786528 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35452fed-bf4d-4760-a3f7-337637d09b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:34.787045 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35452fed-bf4d-4760-a3f7-337637d09b8e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:34.787324 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35452fed-bf4d-4760-a3f7-337637d09b8e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:34.787567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35452fed-bf4d-4760-a3f7-337637d09b8e None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:34.788330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35452fed-bf4d-4760-a3f7-337637d09b8e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:34.788630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35452fed-bf4d-4760-a3f7-337637d09b8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:34.790070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35452fed-bf4d-4760-a3f7-337637d09b8e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:34.809290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 302/628] 10.222.0.36 () {56 vars in 1149 bytes} [Sat Oct 15 13:36:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:34.815068 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44159c3d-bcfb-4966-af73-fd5d5c9cf71c None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:34.868544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:34.933602 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44159c3d-bcfb-4966-af73-fd5d5c9cf71c None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:34.971730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb5c9c-5893-4f4c-abc6-db45a3e9fd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:34.972400 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb5c9c-5893-4f4c-abc6-db45a3e9fd9f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:34.972677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb5c9c-5893-4f4c-abc6-db45a3e9fd9f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:34.973044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb5c9c-5893-4f4c-abc6-db45a3e9fd9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:35.057864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb5c9c-5893-4f4c-abc6-db45a3e9fd9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:35.059278 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb5c9c-5893-4f4c-abc6-db45a3e9fd9f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:35.119316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:35.181305 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 327/629] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:36:34 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 13:36:35.187083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35452fed-bf4d-4760-a3f7-337637d09b8e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:35.239446 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e0a1e-2920-4f4d-adb4-c5e60169d160 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:35.239793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e0a1e-2920-4f4d-adb4-c5e60169d160 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:35.240035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e0a1e-2920-4f4d-adb4-c5e60169d160 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:35.241401 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 328/630] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:35.249571 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7615-0bfe-44a8-a26a-87b2999f295e None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:35.249865 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7615-0bfe-44a8-a26a-87b2999f295e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:35.250119 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada7615-0bfe-44a8-a26a-87b2999f295e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:35.272079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-0ada7615-0bfe-44a8-a26a-87b2999f295e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:35.273328 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0ada7615-0bfe-44a8-a26a-87b2999f295e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:35.299472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:35.341864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35452fed-bf4d-4760-a3f7-337637d09b8e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:35.354920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 329/631] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:36:35.386700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42579ac-00f9-4026-b7fa-1ade22ff44af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:35.387342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42579ac-00f9-4026-b7fa-1ade22ff44af None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:35.387617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42579ac-00f9-4026-b7fa-1ade22ff44af None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:35.387901 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42579ac-00f9-4026-b7fa-1ade22ff44af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:35.400194 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42579ac-00f9-4026-b7fa-1ade22ff44af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:35.402792 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42579ac-00f9-4026-b7fa-1ade22ff44af None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-a42579ac-00f9-4026-b7fa-1ade22ff44af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:35.430063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:35.432430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:35.434524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:35.434524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:35.434524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:35.434524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 303/632] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:35.438063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada7615-0bfe-44a8-a26a-87b2999f295e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:35.525574 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:35.552808 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada7615-0bfe-44a8-a26a-87b2999f295e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:35.604461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4997fa-5d7d-412c-97f4-40348ad627e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:35.605443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4997fa-5d7d-412c-97f4-40348ad627e9 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:35.605868 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4997fa-5d7d-412c-97f4-40348ad627e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:35.606340 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4997fa-5d7d-412c-97f4-40348ad627e9 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:35.607495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4997fa-5d7d-412c-97f4-40348ad627e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:35.607984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4997fa-5d7d-412c-97f4-40348ad627e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:35.611231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4997fa-5d7d-412c-97f4-40348ad627e9 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:35.633519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 330/633] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:35.642086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42579ac-00f9-4026-b7fa-1ade22ff44af None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:35.802051 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4997fa-5d7d-412c-97f4-40348ad627e9 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:35.830577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42579ac-00f9-4026-b7fa-1ade22ff44af None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:35.880114 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:35.889291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719e978-8d7a-456c-9bee-aa1b8b8b73e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:35.890010 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719e978-8d7a-456c-9bee-aa1b8b8b73e1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:35.890961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719e978-8d7a-456c-9bee-aa1b8b8b73e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:35.891248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719e978-8d7a-456c-9bee-aa1b8b8b73e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:35.905720 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c719e978-8d7a-456c-9bee-aa1b8b8b73e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:35.907270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c719e978-8d7a-456c-9bee-aa1b8b8b73e1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-c719e978-8d7a-456c-9bee-aa1b8b8b73e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:35.926105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:35.927745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:35.930215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:35.930215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:35.930215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:35.930215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:35.930215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:35.930215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:35.930215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:35.930215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:35.930215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 304/634] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:35.939913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719e978-8d7a-456c-9bee-aa1b8b8b73e1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:35.968835 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4997fa-5d7d-412c-97f4-40348ad627e9 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:36.030458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b1733-9a50-445b-aaf6-83b754d45201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:36.031836 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b1733-9a50-445b-aaf6-83b754d45201 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:36.032359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b1733-9a50-445b-aaf6-83b754d45201 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:36.032868 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b1733-9a50-445b-aaf6-83b754d45201 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:36.033680 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b1733-9a50-445b-aaf6-83b754d45201 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:36.036138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b1733-9a50-445b-aaf6-83b754d45201 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-df8b1733-9a50-445b-aaf6-83b754d45201 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:36.049229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:36.050346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:36.051170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:36.051170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:36.051170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:36.051170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:36.051170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:36.051170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:36.051170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:36.051170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:36.051170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:36.051170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:36.051170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:36.051170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:36.051170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:36.051170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:36.056055 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 331/635] 10.222.0.36 () {56 vars in 1126 bytes} [Sat Oct 15 13:36:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:36.061143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b1733-9a50-445b-aaf6-83b754d45201 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:36.065391 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:36.141605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719e978-8d7a-456c-9bee-aa1b8b8b73e1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:36.183565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4789776-c550-498e-9b5a-8e9b43189744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:36.184496 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4789776-c550-498e-9b5a-8e9b43189744 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:36.184769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4789776-c550-498e-9b5a-8e9b43189744 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:36.185034 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4789776-c550-498e-9b5a-8e9b43189744 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:36.185740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4789776-c550-498e-9b5a-8e9b43189744 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:36.186044 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4789776-c550-498e-9b5a-8e9b43189744 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:36.187528 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4789776-c550-498e-9b5a-8e9b43189744 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:36.207227 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 305/636] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:36.207227 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4e9ff-2188-45f5-8f8e-9014f8fed504 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:36.207636 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b1733-9a50-445b-aaf6-83b754d45201 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:36.207759 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4e9ff-2188-45f5-8f8e-9014f8fed504 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:36.207759 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4e9ff-2188-45f5-8f8e-9014f8fed504 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:36.208543 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 306/637] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:36.209125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4e9ff-2188-45f5-8f8e-9014f8fed504 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:36.257736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993d0e8-6d0b-4c7a-ae99-3cbf028ceb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:36.258365 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993d0e8-6d0b-4c7a-ae99-3cbf028ceb64 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:36.258641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993d0e8-6d0b-4c7a-ae99-3cbf028ceb64 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:36.258900 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993d0e8-6d0b-4c7a-ae99-3cbf028ceb64 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:36.259585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993d0e8-6d0b-4c7a-ae99-3cbf028ceb64 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:36.259938 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993d0e8-6d0b-4c7a-ae99-3cbf028ceb64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:36.262203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993d0e8-6d0b-4c7a-ae99-3cbf028ceb64 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:36.276138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 332/638] 10.222.0.36 () {56 vars in 1122 bytes} [Sat Oct 15 13:36:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:36.276138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012448f-28a9-4021-8ef5-ae72181b99b2 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:36.276138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012448f-28a9-4021-8ef5-ae72181b99b2 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:36.276138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012448f-28a9-4021-8ef5-ae72181b99b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:36.279300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:36.294890 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-0012448f-28a9-4021-8ef5-ae72181b99b2 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:36.295564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0012448f-28a9-4021-8ef5-ae72181b99b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:36.329320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4e9ff-2188-45f5-8f8e-9014f8fed504 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:36.371300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09846b-0928-4344-a6ec-bc8c6a15787c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:36.371957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09846b-0928-4344-a6ec-bc8c6a15787c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:36.372284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09846b-0928-4344-a6ec-bc8c6a15787c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:36.372559 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09846b-0928-4344-a6ec-bc8c6a15787c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:36.373287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09846b-0928-4344-a6ec-bc8c6a15787c None admin] RBAC: Adding query filter params (role.id=8b837ae2ed57479a93acb638b80e2d67, scope.project.id=2a5a64cc8aa04972bb464407cb0312b4, user.id=2f06388453f44fca9e30fbc37da90bba) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:36.373593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09846b-0928-4344-a6ec-bc8c6a15787c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:36.375712 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09846b-0928-4344-a6ec-bc8c6a15787c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:36.390575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 307/639] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:36:36 2022] GET /identity/v3/role_assignments?user.id=2f06388453f44fca9e30fbc37da90bba&scope.project.id=2a5a64cc8aa04972bb464407cb0312b4&role.id=8b837ae2ed57479a93acb638b80e2d67 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:36.390575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09846b-0928-4344-a6ec-bc8c6a15787c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:36.390894 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:36.414124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 333/640] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:36.581400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09846b-0928-4344-a6ec-bc8c6a15787c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:36.606758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:36.646962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9496f16-223e-4922-b359-c13386db7e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:36.648033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9496f16-223e-4922-b359-c13386db7e0d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:36.648499 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9496f16-223e-4922-b359-c13386db7e0d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:36.648962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9496f16-223e-4922-b359-c13386db7e0d None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:36.650063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9496f16-223e-4922-b359-c13386db7e0d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:36.650655 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9496f16-223e-4922-b359-c13386db7e0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:36.653314 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9496f16-223e-4922-b359-c13386db7e0d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:36.662340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 308/641] 10.222.0.36 () {56 vars in 1122 bytes} [Sat Oct 15 13:36:36 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:36.693566 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012448f-28a9-4021-8ef5-ae72181b99b2 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:36.813245 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012448f-28a9-4021-8ef5-ae72181b99b2 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:36.853188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97633eaa-b30a-4c0b-a65c-0c41facfbfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:36.853830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633eaa-b30a-4c0b-a65c-0c41facfbfcf None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:36.854116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633eaa-b30a-4c0b-a65c-0c41facfbfcf None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:36.854379 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633eaa-b30a-4c0b-a65c-0c41facfbfcf None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:36.855067 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97633eaa-b30a-4c0b-a65c-0c41facfbfcf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:36.856547 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97633eaa-b30a-4c0b-a65c-0c41facfbfcf None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:36.876432 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 334/642] 10.222.0.36 () {60 vars in 1141 bytes} [Sat Oct 15 13:36:36 2022] POST /identity/v3/services => generated 269 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:36:37.204877 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cbfd6-e152-42ac-9cc7-b3b60493c51a None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:37.205412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cbfd6-e152-42ac-9cc7-b3b60493c51a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:37.205807 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cbfd6-e152-42ac-9cc7-b3b60493c51a None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:37.208669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 309/643] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:37.216776 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8943d5-71a7-4bbf-9a52-027aadf2fa4d None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:37.217280 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8943d5-71a7-4bbf-9a52-027aadf2fa4d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:37.217356 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8943d5-71a7-4bbf-9a52-027aadf2fa4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:37.237501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-7a8943d5-71a7-4bbf-9a52-027aadf2fa4d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:37.238727 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7a8943d5-71a7-4bbf-9a52-027aadf2fa4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:37.295461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:37.329815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 335/644] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:37 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:36:37.831659 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cbfd6-e152-42ac-9cc7-b3b60493c51a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:38.015322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cbfd6-e152-42ac-9cc7-b3b60493c51a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:38.051375 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778dbe78-eed9-4e63-ad21-c7d654a8bec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:38.052051 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dbe78-eed9-4e63-ad21-c7d654a8bec9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:38.052324 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dbe78-eed9-4e63-ad21-c7d654a8bec9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:38.052589 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dbe78-eed9-4e63-ad21-c7d654a8bec9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:38.053167 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778dbe78-eed9-4e63-ad21-c7d654a8bec9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:38.054556 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778dbe78-eed9-4e63-ad21-c7d654a8bec9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-778dbe78-eed9-4e63-ad21-c7d654a8bec9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:38.064265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:38.065412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:38.066246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 310/645] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:36:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:38.075473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8943d5-71a7-4bbf-9a52-027aadf2fa4d None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:38.252628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8943d5-71a7-4bbf-9a52-027aadf2fa4d None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:38.311896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3ba67-b47a-4aa5-ae12-3ec42070dff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:38.313252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3ba67-b47a-4aa5-ae12-3ec42070dff0 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:38.313846 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3ba67-b47a-4aa5-ae12-3ec42070dff0 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:38.314380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3ba67-b47a-4aa5-ae12-3ec42070dff0 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:38.315301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3ba67-b47a-4aa5-ae12-3ec42070dff0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:38.315831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3ba67-b47a-4aa5-ae12-3ec42070dff0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:38.317536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3ba67-b47a-4aa5-ae12-3ec42070dff0 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:38.324094 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 336/646] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:36:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:38.333596 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778dbe78-eed9-4e63-ad21-c7d654a8bec9 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:38.414793 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:38.501959 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778dbe78-eed9-4e63-ad21-c7d654a8bec9 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:38.540189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa9a51-9b7d-495e-a2d4-77c938e222d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:38.540697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa9a51-9b7d-495e-a2d4-77c938e222d1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:38.540970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa9a51-9b7d-495e-a2d4-77c938e222d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:38.541221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa9a51-9b7d-495e-a2d4-77c938e222d1 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:38.541851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa9a51-9b7d-495e-a2d4-77c938e222d1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:38.542145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa9a51-9b7d-495e-a2d4-77c938e222d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:38.543539 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa9a51-9b7d-495e-a2d4-77c938e222d1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:38.549033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 311/647] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:36:38 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:38.880149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e54c30-4baa-401e-b6b7-78e887aa03f6 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:38.880814 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e54c30-4baa-401e-b6b7-78e887aa03f6 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:38.881464 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e54c30-4baa-401e-b6b7-78e887aa03f6 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:38.884708 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 337/648] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:38.894281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36f4c9-5374-4bb1-862a-d695213030af None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:38.894843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36f4c9-5374-4bb1-862a-d695213030af None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:38.895065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36f4c9-5374-4bb1-862a-d695213030af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:38.928327 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-3f36f4c9-5374-4bb1-862a-d695213030af None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:38.930163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3f36f4c9-5374-4bb1-862a-d695213030af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:39.062580 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 312/649] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:38 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:39.340334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c3a00-28cd-4d59-96f6-8067cd3fb86b None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:39.340858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c3a00-28cd-4d59-96f6-8067cd3fb86b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:39.340858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c3a00-28cd-4d59-96f6-8067cd3fb86b None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:39.342403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 338/650] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:39.348507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2f266-81de-445a-9283-3b69504a0870 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:39.348800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2f266-81de-445a-9283-3b69504a0870 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:39.349055 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2f266-81de-445a-9283-3b69504a0870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:39.365359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7888d19-39f7-4338-8a9a-2aa72690ad2a None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:39.365985 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7888d19-39f7-4338-8a9a-2aa72690ad2a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:39.366475 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7888d19-39f7-4338-8a9a-2aa72690ad2a None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:39.368386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 339/651] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:36:39.369694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0ff2f266-81de-445a-9283-3b69504a0870 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:39.370302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0ff2f266-81de-445a-9283-3b69504a0870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:39.374778 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2805bb7-16e6-4ed6-b102-0c564ca8566c None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:39.375366 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2805bb7-16e6-4ed6-b102-0c564ca8566c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:39.375917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2805bb7-16e6-4ed6-b102-0c564ca8566c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:39.396121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-b2805bb7-16e6-4ed6-b102-0c564ca8566c None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:39.397796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b2805bb7-16e6-4ed6-b102-0c564ca8566c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:39.399769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:39.445595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 313/652] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:39 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:39.452346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:39.476734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2f266-81de-445a-9283-3b69504a0870 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:39.478284 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 340/653] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:39 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:36:39.589911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2f266-81de-445a-9283-3b69504a0870 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:39.626666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17591771-7232-4452-bfdb-363d79b7320f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:39.627312 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17591771-7232-4452-bfdb-363d79b7320f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:39.627588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17591771-7232-4452-bfdb-363d79b7320f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:39.627850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17591771-7232-4452-bfdb-363d79b7320f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:39.639104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17591771-7232-4452-bfdb-363d79b7320f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-17591771-7232-4452-bfdb-363d79b7320f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:36:39.643999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:39.645075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:39.645976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:39.645976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:39.645976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:39.645976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:39.645976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:36:39.645976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:36:39.645976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:36:39.645976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:36:39.645976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:36:39.645976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:36:39.645976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 13:36:39.645976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:39.645976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 314/654] 10.222.0.36 () {56 vars in 1120 bytes} [Sat Oct 15 13:36:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:36:39.652567 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2805bb7-16e6-4ed6-b102-0c564ca8566c None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:39.718104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17591771-7232-4452-bfdb-363d79b7320f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:39.751788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:39.816173 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2805bb7-16e6-4ed6-b102-0c564ca8566c None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:39.850198 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17591771-7232-4452-bfdb-363d79b7320f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:39.873927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee651389-4977-4a04-858d-42fd73fcc828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:39.874731 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee651389-4977-4a04-858d-42fd73fcc828 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:39.875215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee651389-4977-4a04-858d-42fd73fcc828 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:39.875692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee651389-4977-4a04-858d-42fd73fcc828 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:39.877378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee651389-4977-4a04-858d-42fd73fcc828 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:39.877850 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee651389-4977-4a04-858d-42fd73fcc828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:39.880430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee651389-4977-4a04-858d-42fd73fcc828 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:39.897270 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 341/655] 10.222.0.36 () {56 vars in 1115 bytes} [Sat Oct 15 13:36:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:39.898325 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee651389-4977-4a04-858d-42fd73fcc828 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:39.905862 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb423343-b34c-4833-992f-c3884733f3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:39.906747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb423343-b34c-4833-992f-c3884733f3d0 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:39.906893 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb423343-b34c-4833-992f-c3884733f3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:39.907446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb423343-b34c-4833-992f-c3884733f3d0 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:39.908171 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb423343-b34c-4833-992f-c3884733f3d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:39.909670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb423343-b34c-4833-992f-c3884733f3d0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:39.937362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 315/656] 10.222.0.36 () {60 vars in 1141 bytes} [Sat Oct 15 13:36:39 2022] POST /identity/v3/services => generated 241 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:36:39.938100 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb423343-b34c-4833-992f-c3884733f3d0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:39.975657 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:40.064623 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee651389-4977-4a04-858d-42fd73fcc828 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:40.092451 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:40.135130 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb423343-b34c-4833-992f-c3884733f3d0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:40.139857 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6f4aa-f533-4179-8b21-b576fa5ca043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:40.142363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6f4aa-f533-4179-8b21-b576fa5ca043 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:40.142363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6f4aa-f533-4179-8b21-b576fa5ca043 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:40.142363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6f4aa-f533-4179-8b21-b576fa5ca043 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:40.143114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6f4aa-f533-4179-8b21-b576fa5ca043 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:40.145931 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6f4aa-f533-4179-8b21-b576fa5ca043 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-e3d6f4aa-f533-4179-8b21-b576fa5ca043 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:40.163864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:40.165035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:40.165954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:40.165954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:40.165954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:40.165954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:40.165954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:40.165954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:40.165954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:40.165954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:40.165954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:40.165954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:40.165954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:40.165954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:40.165954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 13:36:40.165954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:40.166694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 342/657] 10.222.0.36 () {56 vars in 1132 bytes} [Sat Oct 15 13:36:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:40.173772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6f4aa-f533-4179-8b21-b576fa5ca043 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:40.203266 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e865e-a540-4552-ac31-aea158f925bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:40.204487 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e865e-a540-4552-ac31-aea158f925bd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:40.204903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e865e-a540-4552-ac31-aea158f925bd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:40.206164 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e865e-a540-4552-ac31-aea158f925bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:40.228274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e865e-a540-4552-ac31-aea158f925bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:40.231008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e865e-a540-4552-ac31-aea158f925bd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-627e865e-a540-4552-ac31-aea158f925bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:40.256456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:40.257509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:40.258403 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:40.258403 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:40.258403 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:40.260943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 316/658] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:40.270022 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e865e-a540-4552-ac31-aea158f925bd None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:40.315080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:40.320203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6f4aa-f533-4179-8b21-b576fa5ca043 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:40.359612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296f254-7a52-48f4-8da9-f0888ad3b4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:40.361418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296f254-7a52-48f4-8da9-f0888ad3b4ca None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:40.361851 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296f254-7a52-48f4-8da9-f0888ad3b4ca None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:40.362264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296f254-7a52-48f4-8da9-f0888ad3b4ca None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:40.363071 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296f254-7a52-48f4-8da9-f0888ad3b4ca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:40.363624 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296f254-7a52-48f4-8da9-f0888ad3b4ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:40.365195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296f254-7a52-48f4-8da9-f0888ad3b4ca None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:40.371242 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 343/659] 10.222.0.36 () {56 vars in 1126 bytes} [Sat Oct 15 13:36:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:40.377668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296f254-7a52-48f4-8da9-f0888ad3b4ca None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:40.390368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e865e-a540-4552-ac31-aea158f925bd None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:40.421892 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:40.427653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de463fb-b5ad-484c-8c34-e7d60c3ad241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:40.428433 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de463fb-b5ad-484c-8c34-e7d60c3ad241 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:40.428830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de463fb-b5ad-484c-8c34-e7d60c3ad241 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:40.429197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de463fb-b5ad-484c-8c34-e7d60c3ad241 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:40.429941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de463fb-b5ad-484c-8c34-e7d60c3ad241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:40.430343 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de463fb-b5ad-484c-8c34-e7d60c3ad241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:40.432064 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de463fb-b5ad-484c-8c34-e7d60c3ad241 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:40.449307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 317/660] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:36:40.454849 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de463fb-b5ad-484c-8c34-e7d60c3ad241 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:40.500404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296f254-7a52-48f4-8da9-f0888ad3b4ca None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:40.539163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd6533-613a-43e8-9797-5dafa7dedf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:40.540305 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd6533-613a-43e8-9797-5dafa7dedf28 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:40.540780 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd6533-613a-43e8-9797-5dafa7dedf28 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:40.541265 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd6533-613a-43e8-9797-5dafa7dedf28 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:40.542156 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd6533-613a-43e8-9797-5dafa7dedf28 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:40.542655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd6533-613a-43e8-9797-5dafa7dedf28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:40.544359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd6533-613a-43e8-9797-5dafa7dedf28 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:40.555606 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 344/661] 10.222.0.36 () {56 vars in 1126 bytes} [Sat Oct 15 13:36:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:36:40.557084 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd6533-613a-43e8-9797-5dafa7dedf28 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:40.565250 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de463fb-b5ad-484c-8c34-e7d60c3ad241 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:40.589229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:40.603031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da574c8-8117-47b0-873d-17590ba9b4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:40.603678 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da574c8-8117-47b0-873d-17590ba9b4e5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:40.603994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da574c8-8117-47b0-873d-17590ba9b4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:40.604266 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da574c8-8117-47b0-873d-17590ba9b4e5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:40.613475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da574c8-8117-47b0-873d-17590ba9b4e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:40.615046 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da574c8-8117-47b0-873d-17590ba9b4e5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1da574c8-8117-47b0-873d-17590ba9b4e5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:40.628784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:40.629808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:40.630745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:40.630745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:40.630745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:40.630745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:40.630745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:40.630745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:40.630745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:40.630745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:40.630745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 13:36:40.630745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:40.634888 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 318/662] 10.222.0.36 () {56 vars in 1117 bytes} [Sat Oct 15 13:36:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:40.635914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da574c8-8117-47b0-873d-17590ba9b4e5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:40.692943 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd6533-613a-43e8-9797-5dafa7dedf28 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:40.695271 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:40.745482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e64c17-0b4e-445a-a298-95fd3db4311f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:40.746338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da574c8-8117-47b0-873d-17590ba9b4e5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:40.746795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e64c17-0b4e-445a-a298-95fd3db4311f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:40.747303 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e64c17-0b4e-445a-a298-95fd3db4311f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:40.748047 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e64c17-0b4e-445a-a298-95fd3db4311f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:40.748855 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e64c17-0b4e-445a-a298-95fd3db4311f None admin] RBAC: Adding query filter params (interface=public, service_id=f47e1bbfbc4a4619bcc5b01f6251c5a5, region_id=RegionOne) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:40.749293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e64c17-0b4e-445a-a298-95fd3db4311f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:40.751508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e64c17-0b4e-445a-a298-95fd3db4311f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:40.759925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 345/663] 10.222.0.36 () {56 vars in 1251 bytes} [Sat Oct 15 13:36:40 2022] GET /identity/v3/endpoints?service_id=f47e1bbfbc4a4619bcc5b01f6251c5a5&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:40.765960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e64c17-0b4e-445a-a298-95fd3db4311f None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:40.792869 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191c4c8-6bd1-4697-a2ef-98532520361f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:40.793850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191c4c8-6bd1-4697-a2ef-98532520361f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:40.794315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191c4c8-6bd1-4697-a2ef-98532520361f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:40.794766 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191c4c8-6bd1-4697-a2ef-98532520361f None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:40.795674 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191c4c8-6bd1-4697-a2ef-98532520361f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:40.796256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191c4c8-6bd1-4697-a2ef-98532520361f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:40.798047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191c4c8-6bd1-4697-a2ef-98532520361f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:40.823006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 319/664] 10.222.0.36 () {56 vars in 1149 bytes} [Sat Oct 15 13:36:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:40.824339 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191c4c8-6bd1-4697-a2ef-98532520361f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:40.899172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:40.915160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e64c17-0b4e-445a-a298-95fd3db4311f None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:40.953394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1068a-44bf-4b8c-ae9e-dad450b2e715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:40.954035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1068a-44bf-4b8c-ae9e-dad450b2e715 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:40.954457 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1068a-44bf-4b8c-ae9e-dad450b2e715 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:40.954628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1068a-44bf-4b8c-ae9e-dad450b2e715 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:40.955396 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec1068a-44bf-4b8c-ae9e-dad450b2e715 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:40.956994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec1068a-44bf-4b8c-ae9e-dad450b2e715 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:40.969107 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 346/665] 10.222.0.36 () {56 vars in 1087 bytes} [Sat Oct 15 13:36:40 2022] GET /identity/v3/services => generated 1258 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:36:40.990760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191c4c8-6bd1-4697-a2ef-98532520361f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:41.005598 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:41.034672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a45e28-8cf8-47ee-8c1c-3a98240a33a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:41.035374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a45e28-8cf8-47ee-8c1c-3a98240a33a2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:41.035720 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a45e28-8cf8-47ee-8c1c-3a98240a33a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:41.036020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a45e28-8cf8-47ee-8c1c-3a98240a33a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:41.049429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a45e28-8cf8-47ee-8c1c-3a98240a33a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:41.050101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a45e28-8cf8-47ee-8c1c-3a98240a33a2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-e3a45e28-8cf8-47ee-8c1c-3a98240a33a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:41.068516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:41.069540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:41.070507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:41.070507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:41.070507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:41.070507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 320/666] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:41.076567 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1068a-44bf-4b8c-ae9e-dad450b2e715 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:41.081849 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddd7fe-bbff-430f-8a99-1aebbda77624 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:41.081849 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddd7fe-bbff-430f-8a99-1aebbda77624 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:41.081849 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddd7fe-bbff-430f-8a99-1aebbda77624 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:41.084503 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 321/667] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:36:41.089115 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123b918-3950-44b3-893c-64557c2a5bec None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:41.089390 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123b918-3950-44b3-893c-64557c2a5bec None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:41.089656 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123b918-3950-44b3-893c-64557c2a5bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:41.114285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c123b918-3950-44b3-893c-64557c2a5bec None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:41.116131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c123b918-3950-44b3-893c-64557c2a5bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:41.197348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1068a-44bf-4b8c-ae9e-dad450b2e715 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:41.204108 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 322/668] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:41 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:36:41.238922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefe107-2260-4f6c-8b62-2f4e787c1be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:41.239864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefe107-2260-4f6c-8b62-2f4e787c1be0 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:41.240383 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefe107-2260-4f6c-8b62-2f4e787c1be0 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:41.240855 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefe107-2260-4f6c-8b62-2f4e787c1be0 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:41.241695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefe107-2260-4f6c-8b62-2f4e787c1be0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:41.242176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefe107-2260-4f6c-8b62-2f4e787c1be0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:41.243940 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaefe107-2260-4f6c-8b62-2f4e787c1be0 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:41.257920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 347/669] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:41.263672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123b918-3950-44b3-893c-64557c2a5bec None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:41.364364 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:41.376093 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123b918-3950-44b3-893c-64557c2a5bec None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:41.415096 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941027b8-7627-4822-83bd-635866d955e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:41.415736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941027b8-7627-4822-83bd-635866d955e0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:41.416061 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941027b8-7627-4822-83bd-635866d955e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:41.416329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941027b8-7627-4822-83bd-635866d955e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:41.424073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941027b8-7627-4822-83bd-635866d955e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:41.425569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941027b8-7627-4822-83bd-635866d955e0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-941027b8-7627-4822-83bd-635866d955e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:41.437413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:41.438411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:41.439841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:41.439841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:41.439841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:41.439841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:41.439841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:41.439841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:41.439841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:41.439841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:41.439841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 323/670] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:41.445565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefe107-2260-4f6c-8b62-2f4e787c1be0 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:41.463899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:41.510066 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941027b8-7627-4822-83bd-635866d955e0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:41.556549 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefe107-2260-4f6c-8b62-2f4e787c1be0 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:41.593233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74552a3e-f979-48ec-9f8c-a33f823e2d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:41.594005 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74552a3e-f979-48ec-9f8c-a33f823e2d0e None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:41.594380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74552a3e-f979-48ec-9f8c-a33f823e2d0e None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:41.594821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74552a3e-f979-48ec-9f8c-a33f823e2d0e None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:41.595729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74552a3e-f979-48ec-9f8c-a33f823e2d0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:41.596166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74552a3e-f979-48ec-9f8c-a33f823e2d0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:41.597712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74552a3e-f979-48ec-9f8c-a33f823e2d0e None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:41.615498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 348/671] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:41.617142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74552a3e-f979-48ec-9f8c-a33f823e2d0e None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:41.636206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941027b8-7627-4822-83bd-635866d955e0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:41.684048 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd34a63-4494-44e7-8eb1-54f916695b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:41.685038 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd34a63-4494-44e7-8eb1-54f916695b58 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:41.685487 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd34a63-4494-44e7-8eb1-54f916695b58 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:41.686190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd34a63-4494-44e7-8eb1-54f916695b58 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:41.687631 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd34a63-4494-44e7-8eb1-54f916695b58 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:41.689962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd34a63-4494-44e7-8eb1-54f916695b58 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:41.697259 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:41.713610 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 324/672] 10.222.0.36 () {60 vars in 1141 bytes} [Sat Oct 15 13:36:41 2022] POST /identity/v3/services => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:36:41.734795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74552a3e-f979-48ec-9f8c-a33f823e2d0e None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:41.788061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b7511-c1b2-478b-a0a8-065acf6739ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:41.789003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b7511-c1b2-478b-a0a8-065acf6739ba None admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:41.789440 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b7511-c1b2-478b-a0a8-065acf6739ba None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:41.789846 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b7511-c1b2-478b-a0a8-065acf6739ba None admin] PATH_INFO: `/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/2f06388453f44fca9e30fbc37da90bba/roles/5bd4f51ce7594d178e68f303bb34cced` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:41.792572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b7511-c1b2-478b-a0a8-065acf6739ba None admin] RBAC: Authorizing `identity:create_grant(project_id=2a5a64cc8aa04972bb464407cb0312b4, user_id=2f06388453f44fca9e30fbc37da90bba, role_id=5bd4f51ce7594d178e68f303bb34cced)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:41.794340 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b7511-c1b2-478b-a0a8-065acf6739ba None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:41.813657 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 349/673] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:36:41 2022] PUT /identity/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/2f06388453f44fca9e30fbc37da90bba/roles/5bd4f51ce7594d178e68f303bb34cced => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:36:42.371973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c978b-90c4-4751-adc0-e96dc9fb4785 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:42.372546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c978b-90c4-4751-adc0-e96dc9fb4785 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:42.372546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c978b-90c4-4751-adc0-e96dc9fb4785 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:42.374154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 325/674] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:42.380450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bb559-7803-421d-9af0-d793c92458f2 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:42.380726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bb559-7803-421d-9af0-d793c92458f2 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:42.381006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bb559-7803-421d-9af0-d793c92458f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:42.400690 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-321bb559-7803-421d-9af0-d793c92458f2 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:42.401899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-321bb559-7803-421d-9af0-d793c92458f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:42.505482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 350/675] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:36:42.770105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c978b-90c4-4751-adc0-e96dc9fb4785 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:42.787245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:42.883900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c978b-90c4-4751-adc0-e96dc9fb4785 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:42.925518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441af30-6f90-4ff0-9310-202d933338b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:42.926308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441af30-6f90-4ff0-9310-202d933338b0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:42.926668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441af30-6f90-4ff0-9310-202d933338b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:42.926938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441af30-6f90-4ff0-9310-202d933338b0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:42.927621 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441af30-6f90-4ff0-9310-202d933338b0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:42.929591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441af30-6f90-4ff0-9310-202d933338b0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f441af30-6f90-4ff0-9310-202d933338b0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:42.941026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:42.942334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:42.943345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 326/676] 10.222.0.36 () {56 vars in 1126 bytes} [Sat Oct 15 13:36:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:42.948640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bb559-7803-421d-9af0-d793c92458f2 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:43.012117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:43.071236 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bb559-7803-421d-9af0-d793c92458f2 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:43.116529 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b25b94-ea7c-44f7-8f2c-d1ce2a2da354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:43.117151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b25b94-ea7c-44f7-8f2c-d1ce2a2da354 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:43.117428 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b25b94-ea7c-44f7-8f2c-d1ce2a2da354 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:43.117697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b25b94-ea7c-44f7-8f2c-d1ce2a2da354 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:43.118340 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b25b94-ea7c-44f7-8f2c-d1ce2a2da354 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:43.118643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b25b94-ea7c-44f7-8f2c-d1ce2a2da354 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:43.120048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b25b94-ea7c-44f7-8f2c-d1ce2a2da354 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:43.130036 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 351/677] 10.222.0.36 () {56 vars in 1122 bytes} [Sat Oct 15 13:36:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:43.136305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441af30-6f90-4ff0-9310-202d933338b0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:43.218313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:43.244702 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441af30-6f90-4ff0-9310-202d933338b0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:43.285891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b9a12-5d17-4dca-88f4-2e11780deaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:43.286513 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b9a12-5d17-4dca-88f4-2e11780deaba None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:43.286786 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b9a12-5d17-4dca-88f4-2e11780deaba None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:43.287053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b9a12-5d17-4dca-88f4-2e11780deaba None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:43.287680 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b9a12-5d17-4dca-88f4-2e11780deaba None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:43.288023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b9a12-5d17-4dca-88f4-2e11780deaba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:43.289406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b9a12-5d17-4dca-88f4-2e11780deaba None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:43.295183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 327/678] 10.222.0.36 () {56 vars in 1122 bytes} [Sat Oct 15 13:36:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:43.301885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b25b94-ea7c-44f7-8f2c-d1ce2a2da354 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:43.456403 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff45b4-c798-4afc-af3d-71c55edbad3f None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:43.457354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff45b4-c798-4afc-af3d-71c55edbad3f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:43.457826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff45b4-c798-4afc-af3d-71c55edbad3f None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:43.459603 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 328/679] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:43.463169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b25b94-ea7c-44f7-8f2c-d1ce2a2da354 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:43.465793 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a2d3f-1b17-422a-8a9e-5a95a4cbfdbd None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:43.466348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a2d3f-1b17-422a-8a9e-5a95a4cbfdbd None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:43.466815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a2d3f-1b17-422a-8a9e-5a95a4cbfdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:43.488686 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b93a2d3f-1b17-422a-8a9e-5a95a4cbfdbd None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:43.490264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b93a2d3f-1b17-422a-8a9e-5a95a4cbfdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:43.506563 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:43.524436 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e13f1b-fbb3-49aa-90bc-7cf12bfc7e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:43.525479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e13f1b-fbb3-49aa-90bc-7cf12bfc7e3f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:43.526057 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e13f1b-fbb3-49aa-90bc-7cf12bfc7e3f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:43.526545 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e13f1b-fbb3-49aa-90bc-7cf12bfc7e3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:43.527921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e13f1b-fbb3-49aa-90bc-7cf12bfc7e3f None admin] RBAC: Adding query filter params (interface=public, service_id=1f8e08ad22cb4abd8d3d86312cc06dca, region_id=RegionOne) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:43.528453 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e13f1b-fbb3-49aa-90bc-7cf12bfc7e3f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:43.531014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e13f1b-fbb3-49aa-90bc-7cf12bfc7e3f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:43.548470 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 352/680] 10.222.0.36 () {56 vars in 1251 bytes} [Sat Oct 15 13:36:43 2022] GET /identity/v3/endpoints?service_id=1f8e08ad22cb4abd8d3d86312cc06dca&interface=public®ion_id=RegionOne => generated 194 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:43.551079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e13f1b-fbb3-49aa-90bc-7cf12bfc7e3f None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:43.573930 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 329/681] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:36:43.714380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e13f1b-fbb3-49aa-90bc-7cf12bfc7e3f None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:43.774635 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d94ad2-9cb2-4c8f-b15f-2210aaa49acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:43.775545 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d94ad2-9cb2-4c8f-b15f-2210aaa49acb None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:43.775986 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d94ad2-9cb2-4c8f-b15f-2210aaa49acb None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:43.776395 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d94ad2-9cb2-4c8f-b15f-2210aaa49acb None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:43.777336 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d94ad2-9cb2-4c8f-b15f-2210aaa49acb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:43.779547 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d94ad2-9cb2-4c8f-b15f-2210aaa49acb None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:43.790967 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 353/682] 10.222.0.36 () {56 vars in 1087 bytes} [Sat Oct 15 13:36:43 2022] GET /identity/v3/services => generated 1497 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:36:43.869697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a2d3f-1b17-422a-8a9e-5a95a4cbfdbd None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:43.972017 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:44.012372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a2d3f-1b17-422a-8a9e-5a95a4cbfdbd None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:44.064562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd3f82-2bb1-477a-89d3-860d7ef0d95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:44.066003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd3f82-2bb1-477a-89d3-860d7ef0d95f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:44.066777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd3f82-2bb1-477a-89d3-860d7ef0d95f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:44.067390 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd3f82-2bb1-477a-89d3-860d7ef0d95f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:44.068458 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd3f82-2bb1-477a-89d3-860d7ef0d95f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:44.070748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd3f82-2bb1-477a-89d3-860d7ef0d95f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-e4bd3f82-2bb1-477a-89d3-860d7ef0d95f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:44.083472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:44.084590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:44.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:44.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:44.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:44.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:44.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:44.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:44.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:44.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:44.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:44.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:44.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:44.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:44.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 15 13:36:44.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:44.088679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 330/683] 10.222.0.36 () {56 vars in 1132 bytes} [Sat Oct 15 13:36:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:44.097159 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d94ad2-9cb2-4c8f-b15f-2210aaa49acb None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:44.187841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f40de-b897-42d6-9084-4b8e2dcd1d35 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:44.189554 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f40de-b897-42d6-9084-4b8e2dcd1d35 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:44.190254 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f40de-b897-42d6-9084-4b8e2dcd1d35 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:44.192521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:44.193262 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 331/684] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:44.195670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e85f7-2478-47e6-bc76-374d652729fc None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:44.195958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e85f7-2478-47e6-bc76-374d652729fc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:44.196395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e85f7-2478-47e6-bc76-374d652729fc None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:44.199761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 332/685] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:44.202211 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6846c9-cf85-4b24-a800-de48a8f3c42a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:44.202816 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6846c9-cf85-4b24-a800-de48a8f3c42a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:44.203281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6846c9-cf85-4b24-a800-de48a8f3c42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:44.230341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2a6846c9-cf85-4b24-a800-de48a8f3c42a None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:44.231501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2a6846c9-cf85-4b24-a800-de48a8f3c42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:44.255539 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d94ad2-9cb2-4c8f-b15f-2210aaa49acb None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:44.293435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0941e-6d5d-4453-a4cb-4a39846e812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:44.294104 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0941e-6d5d-4453-a4cb-4a39846e812b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:44.294534 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0941e-6d5d-4453-a4cb-4a39846e812b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:44.294817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0941e-6d5d-4453-a4cb-4a39846e812b None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:44.295613 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0941e-6d5d-4453-a4cb-4a39846e812b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:44.295943 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0941e-6d5d-4453-a4cb-4a39846e812b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:44.297431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0941e-6d5d-4453-a4cb-4a39846e812b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:44.303388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 354/686] 10.222.0.36 () {56 vars in 1126 bytes} [Sat Oct 15 13:36:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:44.304662 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b69d7-a7e2-482c-b981-25990299a70b None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:44.304945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b69d7-a7e2-482c-b981-25990299a70b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:44.305210 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b69d7-a7e2-482c-b981-25990299a70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:44.324637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-aa5b69d7-a7e2-482c-b981-25990299a70b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:44.325831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-aa5b69d7-a7e2-482c-b981-25990299a70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:44.328836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 333/687] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:36:44.331058 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6846c9-cf85-4b24-a800-de48a8f3c42a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:44.409162 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:44.450204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 355/688] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:44.458739 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6846c9-cf85-4b24-a800-de48a8f3c42a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:44.496631 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78d17c-15fc-4448-b204-65fef94a6502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:44.497196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78d17c-15fc-4448-b204-65fef94a6502 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:44.497461 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78d17c-15fc-4448-b204-65fef94a6502 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:44.497722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78d17c-15fc-4448-b204-65fef94a6502 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:44.498410 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78d17c-15fc-4448-b204-65fef94a6502 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:44.498704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78d17c-15fc-4448-b204-65fef94a6502 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:44.500179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78d17c-15fc-4448-b204-65fef94a6502 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:44.506322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 334/689] 10.222.0.36 () {56 vars in 1126 bytes} [Sat Oct 15 13:36:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:44.512278 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b69d7-a7e2-482c-b981-25990299a70b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:44.589471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:44.635572 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78d17c-15fc-4448-b204-65fef94a6502 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:44.660442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b69d7-a7e2-482c-b981-25990299a70b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:44.711554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efd120-36d2-4925-8c58-7c63b5620475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:44.713315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efd120-36d2-4925-8c58-7c63b5620475 None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:44.714125 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efd120-36d2-4925-8c58-7c63b5620475 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:44.714821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efd120-36d2-4925-8c58-7c63b5620475 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:44.716426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efd120-36d2-4925-8c58-7c63b5620475 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:44.719124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efd120-36d2-4925-8c58-7c63b5620475 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:44.756863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 356/690] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:36:44 2022] POST /identity/v3/endpoints => generated 347 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:36:44.759145 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efd120-36d2-4925-8c58-7c63b5620475 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:44.761441 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78d17c-15fc-4448-b204-65fef94a6502 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:44.795685 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:44.812250 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f470e5-fafb-4417-a6a8-38f69cc4772f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:44.813105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f470e5-fafb-4417-a6a8-38f69cc4772f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:44.813892 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f470e5-fafb-4417-a6a8-38f69cc4772f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:44.814076 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f470e5-fafb-4417-a6a8-38f69cc4772f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:44.826838 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f470e5-fafb-4417-a6a8-38f69cc4772f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-75f470e5-fafb-4417-a6a8-38f69cc4772f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:36:44.832381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:44.834308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:44.836023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:44.836023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:44.836023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:44.836023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:44.836023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:36:44.836023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:36:44.836023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:36:44.836023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:36:44.836023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:36:44.836023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:36:44.836023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 13:36:44.836023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:44.840978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 335/691] 10.222.0.36 () {56 vars in 1120 bytes} [Sat Oct 15 13:36:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:36:44.842504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f470e5-fafb-4417-a6a8-38f69cc4772f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:44.960377 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f470e5-fafb-4417-a6a8-38f69cc4772f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:44.969572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efd120-36d2-4925-8c58-7c63b5620475 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:44.972845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:45.006094 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9160a-d03c-46eb-ae56-0878e232a89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:45.006949 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9160a-d03c-46eb-ae56-0878e232a89a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:45.007243 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9160a-d03c-46eb-ae56-0878e232a89a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:45.007608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9160a-d03c-46eb-ae56-0878e232a89a None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:45.008858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9160a-d03c-46eb-ae56-0878e232a89a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:45.009025 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9160a-d03c-46eb-ae56-0878e232a89a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:45.010510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9160a-d03c-46eb-ae56-0878e232a89a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:45.025501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 336/692] 10.222.0.36 () {56 vars in 1115 bytes} [Sat Oct 15 13:36:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:45.026820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9160a-d03c-46eb-ae56-0878e232a89a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:45.038654 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f6c56-fa16-4d53-b9e1-aee81f00f80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:45.040304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f6c56-fa16-4d53-b9e1-aee81f00f80d None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:45.040959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f6c56-fa16-4d53-b9e1-aee81f00f80d None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:45.041691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f6c56-fa16-4d53-b9e1-aee81f00f80d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:45.042948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f6c56-fa16-4d53-b9e1-aee81f00f80d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:45.045346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f6c56-fa16-4d53-b9e1-aee81f00f80d None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-f70f6c56-fa16-4d53-b9e1-aee81f00f80d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:45.060984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:45.062143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:45.062974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:45.062974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:45.062974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:45.062974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:45.062974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:45.062974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:45.062974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:45.062974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:45.062974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:45.062974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:45.062974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:45.062974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:45.062974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 13:36:45.062974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:45.064485 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 357/693] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:36:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:45.073605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f6c56-fa16-4d53-b9e1-aee81f00f80d None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:45.085832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:45.201699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9160a-d03c-46eb-ae56-0878e232a89a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:45.209403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f6c56-fa16-4d53-b9e1-aee81f00f80d None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:45.248131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394373d1-8ace-4441-9fca-78a96a253479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:45.248820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394373d1-8ace-4441-9fca-78a96a253479 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:45.249086 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394373d1-8ace-4441-9fca-78a96a253479 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:45.249357 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394373d1-8ace-4441-9fca-78a96a253479 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:45.250116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394373d1-8ace-4441-9fca-78a96a253479 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:45.250637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394373d1-8ace-4441-9fca-78a96a253479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:45.252176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394373d1-8ace-4441-9fca-78a96a253479 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:45.252838 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fc88b-f1f8-46dd-a648-821565ab2883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:45.254818 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fc88b-f1f8-46dd-a648-821565ab2883 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:45.255533 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fc88b-f1f8-46dd-a648-821565ab2883 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:45.256387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fc88b-f1f8-46dd-a648-821565ab2883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:45.258553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 358/694] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:36:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:45.263764 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394373d1-8ace-4441-9fca-78a96a253479 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:45.280996 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fc88b-f1f8-46dd-a648-821565ab2883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:45.283995 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fc88b-f1f8-46dd-a648-821565ab2883 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:45.302890 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-a45fc88b-f1f8-46dd-a648-821565ab2883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:45.305789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:45.306756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:45.307645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:45.307645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:45.307645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:45.310068 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 337/695] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:45.323552 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fc88b-f1f8-46dd-a648-821565ab2883 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:45.413713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394373d1-8ace-4441-9fca-78a96a253479 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:45.444286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:45.450810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fc88b-f1f8-46dd-a648-821565ab2883 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:45.463015 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d2e89-f4f4-4beb-a596-f46f8ead99d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:45.463649 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d2e89-f4f4-4beb-a596-f46f8ead99d8 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:45.463996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d2e89-f4f4-4beb-a596-f46f8ead99d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:45.464270 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d2e89-f4f4-4beb-a596-f46f8ead99d8 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:45.464937 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d2e89-f4f4-4beb-a596-f46f8ead99d8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:45.465235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d2e89-f4f4-4beb-a596-f46f8ead99d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:45.466677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d2e89-f4f4-4beb-a596-f46f8ead99d8 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:45.478955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 359/696] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:36:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:45.480488 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d2e89-f4f4-4beb-a596-f46f8ead99d8 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:45.503781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783cccc-e59a-4cc3-9056-d5eaeac7a369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:45.505071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783cccc-e59a-4cc3-9056-d5eaeac7a369 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:45.506412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783cccc-e59a-4cc3-9056-d5eaeac7a369 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:45.506860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783cccc-e59a-4cc3-9056-d5eaeac7a369 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:45.507981 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783cccc-e59a-4cc3-9056-d5eaeac7a369 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:45.508431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783cccc-e59a-4cc3-9056-d5eaeac7a369 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:45.511015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783cccc-e59a-4cc3-9056-d5eaeac7a369 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:45.528391 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 338/697] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:45.538241 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783cccc-e59a-4cc3-9056-d5eaeac7a369 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:45.609316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:45.621117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d2e89-f4f4-4beb-a596-f46f8ead99d8 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:45.668805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0759d3-12e4-49ed-a80a-816219d06abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:45.670701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0759d3-12e4-49ed-a80a-816219d06abe None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:45.671440 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0759d3-12e4-49ed-a80a-816219d06abe None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:45.672258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0759d3-12e4-49ed-a80a-816219d06abe None admin] PATH_INFO: `/v3/endpoints` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:45.673314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0759d3-12e4-49ed-a80a-816219d06abe None admin] RBAC: Adding query filter params (interface=public, service_id=a5d73fc4cb584691808c8d420975a5a4, region_id=RegionOne) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:45.674056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0759d3-12e4-49ed-a80a-816219d06abe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:45.676140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0759d3-12e4-49ed-a80a-816219d06abe None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:45.679771 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783cccc-e59a-4cc3-9056-d5eaeac7a369 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:45.683393 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 360/698] 10.222.0.36 () {56 vars in 1251 bytes} [Sat Oct 15 13:36:45 2022] GET /identity/v3/endpoints?service_id=a5d73fc4cb584691808c8d420975a5a4&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:45.691344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0759d3-12e4-49ed-a80a-816219d06abe None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:45.730569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d421c-0480-49cf-ab94-40bd6392d479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:45.731967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d421c-0480-49cf-ab94-40bd6392d479 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:45.732475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d421c-0480-49cf-ab94-40bd6392d479 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:45.732946 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d421c-0480-49cf-ab94-40bd6392d479 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:45.744188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d421c-0480-49cf-ab94-40bd6392d479 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:45.746776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d421c-0480-49cf-ab94-40bd6392d479 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f40d421c-0480-49cf-ab94-40bd6392d479 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:45.763383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:45.764397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:45.765348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:45.765348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:45.765348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:45.765348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:45.765348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:45.765348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:45.765348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:45.765348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:45.765348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 15 13:36:45.765348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:45.773617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 339/699] 10.222.0.36 () {56 vars in 1117 bytes} [Sat Oct 15 13:36:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:36:45.774950 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d421c-0480-49cf-ab94-40bd6392d479 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:45.806327 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:45.840313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0759d3-12e4-49ed-a80a-816219d06abe None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:45.893006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8ffdc-6160-488b-91a5-45bb0e9acf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:45.893964 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8ffdc-6160-488b-91a5-45bb0e9acf15 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:45.895004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8ffdc-6160-488b-91a5-45bb0e9acf15 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:45.895517 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8ffdc-6160-488b-91a5-45bb0e9acf15 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:45.896825 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8ffdc-6160-488b-91a5-45bb0e9acf15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:45.898728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8ffdc-6160-488b-91a5-45bb0e9acf15 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:45.908355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 361/700] 10.222.0.36 () {56 vars in 1087 bytes} [Sat Oct 15 13:36:45 2022] GET /identity/v3/services => generated 1497 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:36:45.925062 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d421c-0480-49cf-ab94-40bd6392d479 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:45.966682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107de79-f971-4006-87a2-2bb5ddffebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:45.967739 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107de79-f971-4006-87a2-2bb5ddffebef None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:45.968260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107de79-f971-4006-87a2-2bb5ddffebef None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:45.968743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107de79-f971-4006-87a2-2bb5ddffebef None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:45.969736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107de79-f971-4006-87a2-2bb5ddffebef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:45.970260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107de79-f971-4006-87a2-2bb5ddffebef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:45.971978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107de79-f971-4006-87a2-2bb5ddffebef None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:45.995556 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 340/701] 10.222.0.36 () {56 vars in 1149 bytes} [Sat Oct 15 13:36:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:46.002684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8ffdc-6160-488b-91a5-45bb0e9acf15 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:46.043617 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:46.132213 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1b98e-7c54-43b1-b6a4-5bcfac5d7b37 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:46.132927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1b98e-7c54-43b1-b6a4-5bcfac5d7b37 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:46.133265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1b98e-7c54-43b1-b6a4-5bcfac5d7b37 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:46.134925 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 341/702] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:46.141014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c3518-ef10-4f68-8c20-0e344ebe734b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:46.142403 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c3518-ef10-4f68-8c20-0e344ebe734b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:46.142403 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c3518-ef10-4f68-8c20-0e344ebe734b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:46.169203 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6f4c3518-ef10-4f68-8c20-0e344ebe734b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:46.170453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6f4c3518-ef10-4f68-8c20-0e344ebe734b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:46.171512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8ffdc-6160-488b-91a5-45bb0e9acf15 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:46.225224 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03b17b-adab-44b7-8e74-3a99a3aa4844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:46.225852 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03b17b-adab-44b7-8e74-3a99a3aa4844 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:46.226121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03b17b-adab-44b7-8e74-3a99a3aa4844 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:46.226387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03b17b-adab-44b7-8e74-3a99a3aa4844 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:46.259228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 342/703] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:46.310711 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03b17b-adab-44b7-8e74-3a99a3aa4844 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:46.312140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03b17b-adab-44b7-8e74-3a99a3aa4844 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:46.338942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:46.420952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 362/704] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:36:46 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 15 13:36:46.429108 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c3518-ef10-4f68-8c20-0e344ebe734b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:46.438466 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:46.534171 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c3518-ef10-4f68-8c20-0e344ebe734b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:46.568340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8921c0e-2a83-48bb-b082-7e126f675b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:46.568900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8921c0e-2a83-48bb-b082-7e126f675b8d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:46.569177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8921c0e-2a83-48bb-b082-7e126f675b8d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:46.569446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8921c0e-2a83-48bb-b082-7e126f675b8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:46.580086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8921c0e-2a83-48bb-b082-7e126f675b8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:46.581562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8921c0e-2a83-48bb-b082-7e126f675b8d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f8921c0e-2a83-48bb-b082-7e126f675b8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:36:46.596429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:46.597530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:36:46.598406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:36:46.598406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:36:46.598406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:46.598406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 343/705] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:46.599996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03b17b-adab-44b7-8e74-3a99a3aa4844 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:46.609246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8921c0e-2a83-48bb-b082-7e126f675b8d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:46.673838 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:46.712615 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03b17b-adab-44b7-8e74-3a99a3aa4844 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:46.719249 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8921c0e-2a83-48bb-b082-7e126f675b8d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:46.739732 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:46.753909 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136cf90-6853-4fa3-b37c-7e5252576f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:46.755011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136cf90-6853-4fa3-b37c-7e5252576f87 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:46.755557 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136cf90-6853-4fa3-b37c-7e5252576f87 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:46.756212 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136cf90-6853-4fa3-b37c-7e5252576f87 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:46.757156 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136cf90-6853-4fa3-b37c-7e5252576f87 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:46.759115 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136cf90-6853-4fa3-b37c-7e5252576f87 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:46.759411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da6001-06e9-48dc-9ea7-b799e2fb228f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:46.760061 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da6001-06e9-48dc-9ea7-b799e2fb228f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:46.760339 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da6001-06e9-48dc-9ea7-b799e2fb228f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:46.760605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da6001-06e9-48dc-9ea7-b799e2fb228f None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:46.761257 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da6001-06e9-48dc-9ea7-b799e2fb228f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:46.761554 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da6001-06e9-48dc-9ea7-b799e2fb228f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:46.763150 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da6001-06e9-48dc-9ea7-b799e2fb228f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-c136cf90-6853-4fa3-b37c-7e5252576f87 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:46.770996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:46.772106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:46.772982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:46.772982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:46.772982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:46.772982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:46.772982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:46.772982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:46.772982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:46.772982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:46.772982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:46.772982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:46.772982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:46.772982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:46.772982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:46.772982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:46.779965 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 344/706] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:46.782724 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da6001-06e9-48dc-9ea7-b799e2fb228f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:46.784825 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 363/707] 10.222.0.36 () {56 vars in 1126 bytes} [Sat Oct 15 13:36:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:46.786185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136cf90-6853-4fa3-b37c-7e5252576f87 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:46.918262 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136cf90-6853-4fa3-b37c-7e5252576f87 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:46.927248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da6001-06e9-48dc-9ea7-b799e2fb228f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:46.951682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:46.964518 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3baedc-db4e-427e-b762-dd94bac505e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:46.965597 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3baedc-db4e-427e-b762-dd94bac505e6 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:46.966178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3baedc-db4e-427e-b762-dd94bac505e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:46.966656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3baedc-db4e-427e-b762-dd94bac505e6 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:46.967570 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3baedc-db4e-427e-b762-dd94bac505e6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:46.968233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3baedc-db4e-427e-b762-dd94bac505e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:46.970200 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3baedc-db4e-427e-b762-dd94bac505e6 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:46.971738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52138305-8fab-4888-9e6a-7d7f902b8d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:46.972536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52138305-8fab-4888-9e6a-7d7f902b8d94 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:46.973273 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52138305-8fab-4888-9e6a-7d7f902b8d94 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:46.973641 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52138305-8fab-4888-9e6a-7d7f902b8d94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:46.976662 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 364/708] 10.222.0.36 () {56 vars in 1122 bytes} [Sat Oct 15 13:36:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:46.985927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3baedc-db4e-427e-b762-dd94bac505e6 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:46.989138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52138305-8fab-4888-9e6a-7d7f902b8d94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:46.991045 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52138305-8fab-4888-9e6a-7d7f902b8d94 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-52138305-8fab-4888-9e6a-7d7f902b8d94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:47.004617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:47.005553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:47.006409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:47.006409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:47.006409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:47.006409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:47.006409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:47.006409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:47.006409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:47.006409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:47.013582 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 345/709] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:47.014776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52138305-8fab-4888-9e6a-7d7f902b8d94 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:47.046167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:47.122259 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3baedc-db4e-427e-b762-dd94bac505e6 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:47.133813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52138305-8fab-4888-9e6a-7d7f902b8d94 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:47.167618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a0d94-4b78-472c-83d5-79b6f34074d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:47.168341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a0d94-4b78-472c-83d5-79b6f34074d4 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:47.168621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a0d94-4b78-472c-83d5-79b6f34074d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:47.168898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a0d94-4b78-472c-83d5-79b6f34074d4 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:47.169607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a0d94-4b78-472c-83d5-79b6f34074d4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:47.169919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a0d94-4b78-472c-83d5-79b6f34074d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:47.171497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a0d94-4b78-472c-83d5-79b6f34074d4 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:47.179118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 365/710] 10.222.0.36 () {56 vars in 1122 bytes} [Sat Oct 15 13:36:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:36:47.183017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e7c3d-fddb-4400-a4b5-cc7ce1cfae66 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:47.183577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e7c3d-fddb-4400-a4b5-cc7ce1cfae66 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:47.183577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e7c3d-fddb-4400-a4b5-cc7ce1cfae66 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:47.186731 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b3a8c-33b2-41d5-8f83-28a487eeb314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:47.187670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b3a8c-33b2-41d5-8f83-28a487eeb314 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:47.187980 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b3a8c-33b2-41d5-8f83-28a487eeb314 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:47.188185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 366/711] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:36:47.188283 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b3a8c-33b2-41d5-8f83-28a487eeb314 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:47.189322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b3a8c-33b2-41d5-8f83-28a487eeb314 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:47.189623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b3a8c-33b2-41d5-8f83-28a487eeb314 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:47.189958 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e7c3d-fddb-4400-a4b5-cc7ce1cfae66 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:47.191094 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b3a8c-33b2-41d5-8f83-28a487eeb314 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:47.208445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 346/712] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:36:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:47.208445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b976c-ef02-4de8-9c4b-714c649e7e82 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:47.208445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b976c-ef02-4de8-9c4b-714c649e7e82 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:47.208445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b976c-ef02-4de8-9c4b-714c649e7e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:47.229777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-836b976c-ef02-4de8-9c4b-714c649e7e82 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:47.231496 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-836b976c-ef02-4de8-9c4b-714c649e7e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:47.271028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:47.309147 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e7c3d-fddb-4400-a4b5-cc7ce1cfae66 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:47.319137 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 347/713] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:47.320275 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b976c-ef02-4de8-9c4b-714c649e7e82 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:47.324685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:47.346308 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4308d-0159-4487-968f-7d32fe6d9d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:47.346925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4308d-0159-4487-968f-7d32fe6d9d86 None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:47.347204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4308d-0159-4487-968f-7d32fe6d9d86 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:47.347473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4308d-0159-4487-968f-7d32fe6d9d86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:47.348167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4308d-0159-4487-968f-7d32fe6d9d86 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:47.349563 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4308d-0159-4487-968f-7d32fe6d9d86 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:47.380772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 367/714] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:36:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:36:47.461913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b976c-ef02-4de8-9c4b-714c649e7e82 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:47.510430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9b6f8-1147-49e7-82f5-86d359e786df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:47.511488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9b6f8-1147-49e7-82f5-86d359e786df None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:47.511923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9b6f8-1147-49e7-82f5-86d359e786df None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:47.512329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9b6f8-1147-49e7-82f5-86d359e786df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:47.513268 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9b6f8-1147-49e7-82f5-86d359e786df None admin] RBAC: Adding query filter params (role.id=5bd4f51ce7594d178e68f303bb34cced, scope.project.id=2a5a64cc8aa04972bb464407cb0312b4, user.id=2f06388453f44fca9e30fbc37da90bba) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:47.513820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9b6f8-1147-49e7-82f5-86d359e786df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:47.515449 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9b6f8-1147-49e7-82f5-86d359e786df None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:47.523258 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 348/715] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:36:47 2022] GET /identity/v3/role_assignments?user.id=2f06388453f44fca9e30fbc37da90bba&scope.project.id=2a5a64cc8aa04972bb464407cb0312b4&role.id=5bd4f51ce7594d178e68f303bb34cced => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:47.600406 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4308d-0159-4487-968f-7d32fe6d9d86 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:47.713824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4308d-0159-4487-968f-7d32fe6d9d86 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:47.751622 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f88df-f5a2-4a74-8633-68e3024740e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:47.752328 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f88df-f5a2-4a74-8633-68e3024740e7 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:47.752609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f88df-f5a2-4a74-8633-68e3024740e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:47.752878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f88df-f5a2-4a74-8633-68e3024740e7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:47.753476 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f88df-f5a2-4a74-8633-68e3024740e7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:47.754871 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f88df-f5a2-4a74-8633-68e3024740e7 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-2a6f88df-f5a2-4a74-8633-68e3024740e7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:47.764494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:47.765537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:47.766363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 368/716] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:47.774235 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9b6f8-1147-49e7-82f5-86d359e786df None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:47.809162 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:47.894485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9b6f8-1147-49e7-82f5-86d359e786df None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:47.931733 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60602d8c-091f-4b29-821b-3715e52bbda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:47.932382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60602d8c-091f-4b29-821b-3715e52bbda7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:47.932675 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60602d8c-091f-4b29-821b-3715e52bbda7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:47.932938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60602d8c-091f-4b29-821b-3715e52bbda7 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:47.933565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60602d8c-091f-4b29-821b-3715e52bbda7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:47.933858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60602d8c-091f-4b29-821b-3715e52bbda7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:47.935246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60602d8c-091f-4b29-821b-3715e52bbda7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:47.942550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 349/717] 10.222.0.36 () {56 vars in 1112 bytes} [Sat Oct 15 13:36:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:47.946897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f88df-f5a2-4a74-8633-68e3024740e7 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:47.987198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:48.069886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f88df-f5a2-4a74-8633-68e3024740e7 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:48.112352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daefa56-7694-4cc9-b439-58935e9fbbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:48.113004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daefa56-7694-4cc9-b439-58935e9fbbf8 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:48.113274 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daefa56-7694-4cc9-b439-58935e9fbbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:48.113619 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daefa56-7694-4cc9-b439-58935e9fbbf8 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:48.114297 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daefa56-7694-4cc9-b439-58935e9fbbf8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:48.114599 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daefa56-7694-4cc9-b439-58935e9fbbf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:48.116111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daefa56-7694-4cc9-b439-58935e9fbbf8 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:48.122263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 369/718] 10.222.0.36 () {56 vars in 1112 bytes} [Sat Oct 15 13:36:47 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:48.227791 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed29d9-36ef-45d7-be26-284be41413f1 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:48.228156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed29d9-36ef-45d7-be26-284be41413f1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:48.228387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed29d9-36ef-45d7-be26-284be41413f1 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:48.230011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 350/719] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:48.236189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07195bde-8287-4d84-a896-572be856927b None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:48.236881 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07195bde-8287-4d84-a896-572be856927b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:48.237412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07195bde-8287-4d84-a896-572be856927b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:48.257595 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-07195bde-8287-4d84-a896-572be856927b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:48.259191 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-07195bde-8287-4d84-a896-572be856927b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:48.343899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 370/720] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:48.601985 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed29d9-36ef-45d7-be26-284be41413f1 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:48.705664 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:48.716067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed29d9-36ef-45d7-be26-284be41413f1 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:48.756747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503cecc-6f8e-496e-9c6c-7efbc8dec65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:48.757798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503cecc-6f8e-496e-9c6c-7efbc8dec65e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:48.758275 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503cecc-6f8e-496e-9c6c-7efbc8dec65e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:48.758740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503cecc-6f8e-496e-9c6c-7efbc8dec65e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:48.759527 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503cecc-6f8e-496e-9c6c-7efbc8dec65e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:48.761156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503cecc-6f8e-496e-9c6c-7efbc8dec65e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-e503cecc-6f8e-496e-9c6c-7efbc8dec65e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:48.770882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:48.771911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:48.772748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:48.772748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:48.772748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:48.772748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:48.772748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:48.772748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:48.772748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:48.772748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:48.772748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:48.772748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:48.772748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:48.772748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:48.772748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 15 13:36:48.772748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:48.775499 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 351/721] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:36:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:48.782269 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07195bde-8287-4d84-a896-572be856927b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:48.813018 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:48.885063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07195bde-8287-4d84-a896-572be856927b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:48.923724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8400e-fbc6-4327-a29d-e20f35e5171c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:48.924384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8400e-fbc6-4327-a29d-e20f35e5171c None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:48.924663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8400e-fbc6-4327-a29d-e20f35e5171c None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:48.924934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8400e-fbc6-4327-a29d-e20f35e5171c None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:48.925587 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8400e-fbc6-4327-a29d-e20f35e5171c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:48.925887 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8400e-fbc6-4327-a29d-e20f35e5171c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:48.927259 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8400e-fbc6-4327-a29d-e20f35e5171c None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:48.932798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 371/722] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:36:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:48.938590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503cecc-6f8e-496e-9c6c-7efbc8dec65e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:49.046675 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503cecc-6f8e-496e-9c6c-7efbc8dec65e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:49.082510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d41020-8bd2-4fa7-bfb0-7fca30dd4342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:49.083140 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d41020-8bd2-4fa7-bfb0-7fca30dd4342 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:49.083413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d41020-8bd2-4fa7-bfb0-7fca30dd4342 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:49.083679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d41020-8bd2-4fa7-bfb0-7fca30dd4342 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:49.084356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d41020-8bd2-4fa7-bfb0-7fca30dd4342 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:49.084655 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d41020-8bd2-4fa7-bfb0-7fca30dd4342 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:49.086083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d41020-8bd2-4fa7-bfb0-7fca30dd4342 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:49.092420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 352/723] 10.222.0.36 () {56 vars in 1114 bytes} [Sat Oct 15 13:36:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:49.098740 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8400e-fbc6-4327-a29d-e20f35e5171c None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:49.201147 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:49.211736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8400e-fbc6-4327-a29d-e20f35e5171c None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:49.247589 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f354c-d10d-4825-a0fe-166a758346d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:49.248264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f354c-d10d-4825-a0fe-166a758346d7 None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:49.248540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f354c-d10d-4825-a0fe-166a758346d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:49.248814 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f354c-d10d-4825-a0fe-166a758346d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:49.249500 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f354c-d10d-4825-a0fe-166a758346d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:49.251204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f354c-d10d-4825-a0fe-166a758346d7 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:49.274440 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 372/724] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:36:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:36:49.690757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce731f17-c23c-4d58-8c64-a0333d8da428 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:49.691174 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce731f17-c23c-4d58-8c64-a0333d8da428 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:49.691350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce731f17-c23c-4d58-8c64-a0333d8da428 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:49.692922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 353/725] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:49.698618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effaa6d-6ec1-4461-846a-2896ffbe2f66 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:49.698909 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effaa6d-6ec1-4461-846a-2896ffbe2f66 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:49.699188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effaa6d-6ec1-4461-846a-2896ffbe2f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:49.718054 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-0effaa6d-6ec1-4461-846a-2896ffbe2f66 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:49.719253 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0effaa6d-6ec1-4461-846a-2896ffbe2f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:49.802331 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 373/726] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:49.883955 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278c61b-0c5a-4b68-beb3-b86f128aff41 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:49.890388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278c61b-0c5a-4b68-beb3-b86f128aff41 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:49.890859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278c61b-0c5a-4b68-beb3-b86f128aff41 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:49.893054 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 354/727] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:49 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:49.898910 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a966594-c121-4d44-a6e8-e78cbf6231e8 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:49.899438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a966594-c121-4d44-a6e8-e78cbf6231e8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:49.899927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a966594-c121-4d44-a6e8-e78cbf6231e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:49.918922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-6a966594-c121-4d44-a6e8-e78cbf6231e8 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:49.920518 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6a966594-c121-4d44-a6e8-e78cbf6231e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:49.997131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 374/728] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:50.076363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278c61b-0c5a-4b68-beb3-b86f128aff41 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:50.103030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:50.188178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278c61b-0c5a-4b68-beb3-b86f128aff41 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:50.228242 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca798c6-8f52-45fa-9227-0fb36770be82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:50.229297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca798c6-8f52-45fa-9227-0fb36770be82 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:50.229774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca798c6-8f52-45fa-9227-0fb36770be82 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:50.230255 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca798c6-8f52-45fa-9227-0fb36770be82 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:50.231031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca798c6-8f52-45fa-9227-0fb36770be82 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:50.232733 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca798c6-8f52-45fa-9227-0fb36770be82 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-fca798c6-8f52-45fa-9227-0fb36770be82 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:50.243418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:50.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:50.245335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:50.245335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:50.245335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:50.245335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:50.245335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:50.245335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:50.245335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:50.245335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:50.245335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:50.245335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:50.245335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:50.245335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:50.245335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:50.245335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:50.248181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 355/729] 10.222.0.36 () {56 vars in 1126 bytes} [Sat Oct 15 13:36:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:50.257047 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a966594-c121-4d44-a6e8-e78cbf6231e8 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:50.261895 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:50.270957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca798c6-8f52-45fa-9227-0fb36770be82 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:50.360475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:50.376047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca798c6-8f52-45fa-9227-0fb36770be82 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:50.405438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a966594-c121-4d44-a6e8-e78cbf6231e8 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:50.415601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1215d-e29a-42de-b616-d6ac0c2216c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:50.416278 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1215d-e29a-42de-b616-d6ac0c2216c0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:50.416547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1215d-e29a-42de-b616-d6ac0c2216c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:50.416809 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1215d-e29a-42de-b616-d6ac0c2216c0 None admin] PATH_INFO: `/v3/services/image` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:50.417485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1215d-e29a-42de-b616-d6ac0c2216c0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:50.418971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1215d-e29a-42de-b616-d6ac0c2216c0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-8ab1215d-e29a-42de-b616-d6ac0c2216c0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:50.428762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:50.429788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:50.430607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:50.430607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:50.430607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:50.430607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:50.430607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:50.430607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:50.430607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:50.430607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:50.430607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:50.430607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:50.430607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:50.430607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:50.430607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 13:36:50.430607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:50.435005 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 356/730] 10.222.0.36 () {56 vars in 1105 bytes} [Sat Oct 15 13:36:50 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:36:50.435988 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1215d-e29a-42de-b616-d6ac0c2216c0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:50.448910 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2699d6-4ad8-47f6-99d0-39a225a9aa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:50.449528 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2699d6-4ad8-47f6-99d0-39a225a9aa40 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:50.449799 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2699d6-4ad8-47f6-99d0-39a225a9aa40 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:50.450065 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2699d6-4ad8-47f6-99d0-39a225a9aa40 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:50.450800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2699d6-4ad8-47f6-99d0-39a225a9aa40 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:50.451101 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2699d6-4ad8-47f6-99d0-39a225a9aa40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:50.452506 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2699d6-4ad8-47f6-99d0-39a225a9aa40 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:50.462808 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 375/731] 10.222.0.36 () {56 vars in 1122 bytes} [Sat Oct 15 13:36:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:50.463565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2699d6-4ad8-47f6-99d0-39a225a9aa40 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:50.543761 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:50.546797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1215d-e29a-42de-b616-d6ac0c2216c0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:50.579537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2699d6-4ad8-47f6-99d0-39a225a9aa40 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:50.585323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c6ca2-b3b9-41c4-8104-f59a63c9d4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:50.586039 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c6ca2-b3b9-41c4-8104-f59a63c9d4dd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:50.586284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c6ca2-b3b9-41c4-8104-f59a63c9d4dd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:50.586591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c6ca2-b3b9-41c4-8104-f59a63c9d4dd None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:50.587752 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c6ca2-b3b9-41c4-8104-f59a63c9d4dd None admin] RBAC: Adding query filter params (name=image) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:50.588083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c6ca2-b3b9-41c4-8104-f59a63c9d4dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:50.589927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c6ca2-b3b9-41c4-8104-f59a63c9d4dd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:50.602674 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 357/732] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:50.602674 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7096d-e62b-4086-bd71-1baefcda5c38 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:50.602674 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7096d-e62b-4086-bd71-1baefcda5c38 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:50.602674 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7096d-e62b-4086-bd71-1baefcda5c38 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:50.603441 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 358/733] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:50.604469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7096d-e62b-4086-bd71-1baefcda5c38 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:50.619377 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8fbbc0-9a75-40a1-9d63-8af05365a629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:50.620417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fbbc0-9a75-40a1-9d63-8af05365a629 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:50.620887 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fbbc0-9a75-40a1-9d63-8af05365a629 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:50.621398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fbbc0-9a75-40a1-9d63-8af05365a629 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:50.621601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:50.622131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8fbbc0-9a75-40a1-9d63-8af05365a629 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:50.622504 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8fbbc0-9a75-40a1-9d63-8af05365a629 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:50.624460 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8fbbc0-9a75-40a1-9d63-8af05365a629 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:50.635490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 376/734] 10.222.0.36 () {56 vars in 1122 bytes} [Sat Oct 15 13:36:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:50.635490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee45bad-ee62-4d35-83e4-d0ec4648785e None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:50.635490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee45bad-ee62-4d35-83e4-d0ec4648785e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:50.635490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee45bad-ee62-4d35-83e4-d0ec4648785e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:50.652696 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-8ee45bad-ee62-4d35-83e4-d0ec4648785e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:50.653871 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8ee45bad-ee62-4d35-83e4-d0ec4648785e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:50.728100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 377/735] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:50.728100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee45bad-ee62-4d35-83e4-d0ec4648785e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:50.737723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7096d-e62b-4086-bd71-1baefcda5c38 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:50.777459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26228887-8816-405e-ba55-0d55200371bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:50.778075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26228887-8816-405e-ba55-0d55200371bd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:50.778347 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26228887-8816-405e-ba55-0d55200371bd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:50.778611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26228887-8816-405e-ba55-0d55200371bd None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:50.779233 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26228887-8816-405e-ba55-0d55200371bd None admin] RBAC: Adding query filter params (type=image) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:50.779529 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26228887-8816-405e-ba55-0d55200371bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:50.780957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26228887-8816-405e-ba55-0d55200371bd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:50.787561 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 359/736] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:50.826791 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:50.875162 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee45bad-ee62-4d35-83e4-d0ec4648785e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:50.914186 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681773b-4194-441f-9ae9-62b6cf9f6075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:50.914818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681773b-4194-441f-9ae9-62b6cf9f6075 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:50.915091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681773b-4194-441f-9ae9-62b6cf9f6075 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:50.915356 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681773b-4194-441f-9ae9-62b6cf9f6075 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:50.916117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681773b-4194-441f-9ae9-62b6cf9f6075 None admin] RBAC: Adding query filter params (interface=admin, service_id=1f8e08ad22cb4abd8d3d86312cc06dca, region_id=RegionOne) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:50.916424 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681773b-4194-441f-9ae9-62b6cf9f6075 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:50.917869 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681773b-4194-441f-9ae9-62b6cf9f6075 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:50.925389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 378/737] 10.222.0.36 () {56 vars in 1249 bytes} [Sat Oct 15 13:36:50 2022] GET /identity/v3/endpoints?service_id=1f8e08ad22cb4abd8d3d86312cc06dca&interface=admin®ion_id=RegionOne => generated 193 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:50.931288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26228887-8816-405e-ba55-0d55200371bd None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:51.009713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681773b-4194-441f-9ae9-62b6cf9f6075 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:51.034986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:51.064966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26228887-8816-405e-ba55-0d55200371bd None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:51.103833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a39f36-2f40-4c73-8f68-ea8cfe328ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:51.104699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a39f36-2f40-4c73-8f68-ea8cfe328ded None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:51.105122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a39f36-2f40-4c73-8f68-ea8cfe328ded None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:51.105440 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a39f36-2f40-4c73-8f68-ea8cfe328ded None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:51.106051 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a39f36-2f40-4c73-8f68-ea8cfe328ded None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:51.107646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a39f36-2f40-4c73-8f68-ea8cfe328ded None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:51.122544 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 360/738] 10.222.0.36 () {56 vars in 1087 bytes} [Sat Oct 15 13:36:50 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:36:51.137015 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681773b-4194-441f-9ae9-62b6cf9f6075 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:51.168878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:51.179392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172417ef-b356-444b-9ebf-48095ad3e08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:51.180466 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172417ef-b356-444b-9ebf-48095ad3e08b None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:51.181023 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172417ef-b356-444b-9ebf-48095ad3e08b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:51.181511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172417ef-b356-444b-9ebf-48095ad3e08b None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:51.182411 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172417ef-b356-444b-9ebf-48095ad3e08b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:51.184082 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172417ef-b356-444b-9ebf-48095ad3e08b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:51.204357 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 379/739] 10.222.0.36 () {60 vars in 1141 bytes} [Sat Oct 15 13:36:51 2022] POST /identity/v3/services => generated 242 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:36:51.529951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962d561-44bd-4deb-935f-7e7028a5e4b1 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:51.530131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962d561-44bd-4deb-935f-7e7028a5e4b1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:51.530442 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962d561-44bd-4deb-935f-7e7028a5e4b1 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:51.532179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 361/740] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:51.538098 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8464b-8658-4513-acfd-4a364077e07e None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:51.538386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8464b-8658-4513-acfd-4a364077e07e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:51.538648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8464b-8658-4513-acfd-4a364077e07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:51.557445 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-46e8464b-8658-4513-acfd-4a364077e07e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:51.558637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-46e8464b-8658-4513-acfd-4a364077e07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:51.640042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 380/741] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:51.909490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962d561-44bd-4deb-935f-7e7028a5e4b1 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:52.014009 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962d561-44bd-4deb-935f-7e7028a5e4b1 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:52.047651 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f70535-20fa-402b-abc6-8fb2f1b93aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:52.048385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f70535-20fa-402b-abc6-8fb2f1b93aa5 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:52.048708 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f70535-20fa-402b-abc6-8fb2f1b93aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:52.049015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f70535-20fa-402b-abc6-8fb2f1b93aa5 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:52.049676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f70535-20fa-402b-abc6-8fb2f1b93aa5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:52.051123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f70535-20fa-402b-abc6-8fb2f1b93aa5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:52.053903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: WARNING keystone.server.flask.common [None req-11f70535-20fa-402b-abc6-8fb2f1b93aa5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 15 13:36:52.065587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:52.075925 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.sql.core [None req-11f70535-20fa-402b-abc6-8fb2f1b93aa5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 15 13:36:52.075925 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 13:36:52.075925 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [parameters: {'id': 'b9c9ff0eb752437fa41e9bdd2728b833', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 15 13:36:52.075925 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87916) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 13:36:52.085228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-11f70535-20fa-402b-abc6-8fb2f1b93aa5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b9c9ff0eb752437fa41e9bdd2728b833.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b9c9ff0eb752437fa41e9bdd2728b833. Oct 15 13:36:52.085228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:52.085228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 13:36:52.085228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 13:36:52.085228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 13:36:52.085228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 13:36:52.085228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 13:36:52.085228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 13:36:52.085228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 13:36:52.085228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application conn.query(q) Oct 15 13:36:52.085228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 13:36:52.085228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 13:36:52.085228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result.read() Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 15 13:36:52.086109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application next(self.gen) Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application yield resource Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application next(self.gen) Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise self.value Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application session.commit() Oct 15 13:36:52.087029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.session.flush() Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application compat.raise_( Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 13:36:52.087913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 13:36:52.088800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application conn.query(q) Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result.read() Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 13:36:52.089567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [parameters: {'id': 'b9c9ff0eb752437fa41e9bdd2728b833', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:52.091941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:52.094035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:52.100073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:52.100073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 15 13:36:52.100073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 15 13:36:52.100073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:52.100073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:52.100073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 15 13:36:52.100073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 15 13:36:52.100073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b9c9ff0eb752437fa41e9bdd2728b833. Oct 15 13:36:52.100073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:52.100073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 362/742] 10.222.0.36 () {60 vars in 1140 bytes} [Sat Oct 15 13:36:51 2022] POST /identity/v3/projects => generated 262 bytes in 178 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:52.100652 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8464b-8658-4513-acfd-4a364077e07e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:52.205789 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8464b-8658-4513-acfd-4a364077e07e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:52.220309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:52.243355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d0e84-7b0f-4783-8819-b14a080947b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:52.244383 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d0e84-7b0f-4783-8819-b14a080947b0 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:52.244850 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d0e84-7b0f-4783-8819-b14a080947b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:52.245274 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d0e84-7b0f-4783-8819-b14a080947b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:52.252655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d0e84-7b0f-4783-8819-b14a080947b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:52.254311 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d0e84-7b0f-4783-8819-b14a080947b0 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-759d0e84-7b0f-4783-8819-b14a080947b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:36:52.266003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:52.267148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:52.268252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:36:52.268252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:36:52.268252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:36:52.268252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:36:52.268252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:36:52.268252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:36:52.268252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:36:52.268252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:52.274917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 381/743] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:52.276825 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f70535-20fa-402b-abc6-8fb2f1b93aa5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:52.386097 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f70535-20fa-402b-abc6-8fb2f1b93aa5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:52.420777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f93c65-2e3f-4ac0-971d-1d1daca01d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:52.421395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f93c65-2e3f-4ac0-971d-1d1daca01d26 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:52.421667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f93c65-2e3f-4ac0-971d-1d1daca01d26 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:52.421931 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f93c65-2e3f-4ac0-971d-1d1daca01d26 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:52.422590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f93c65-2e3f-4ac0-971d-1d1daca01d26 None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:52.423163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f93c65-2e3f-4ac0-971d-1d1daca01d26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:52.424916 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f93c65-2e3f-4ac0-971d-1d1daca01d26 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:52.445947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 363/744] 10.222.0.36 () {56 vars in 1112 bytes} [Sat Oct 15 13:36:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:53.172217 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b30b19-f6be-4d72-9b7a-f3efd1a59484 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:53.172807 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b30b19-f6be-4d72-9b7a-f3efd1a59484 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:53.172807 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b30b19-f6be-4d72-9b7a-f3efd1a59484 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:53.174866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 382/745] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:53.180761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203ec2c-cde9-4ce0-9c03-d0f154054918 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:53.181250 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203ec2c-cde9-4ce0-9c03-d0f154054918 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:53.181327 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203ec2c-cde9-4ce0-9c03-d0f154054918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:53.201258 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f203ec2c-cde9-4ce0-9c03-d0f154054918 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:53.202775 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f203ec2c-cde9-4ce0-9c03-d0f154054918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:53.262114 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:53.283029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 364/746] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:53.390832 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7963a2e-74c1-44d6-b147-e071d628917c None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:53.391181 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7963a2e-74c1-44d6-b147-e071d628917c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:53.391370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7963a2e-74c1-44d6-b147-e071d628917c None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:53.393582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 383/747] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:53.399850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf04d2-5275-4b5d-8c44-f2f746c7303b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:53.400511 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf04d2-5275-4b5d-8c44-f2f746c7303b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:53.400976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf04d2-5275-4b5d-8c44-f2f746c7303b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:53.420208 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ecaf04d2-5275-4b5d-8c44-f2f746c7303b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:53.421716 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ecaf04d2-5275-4b5d-8c44-f2f746c7303b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:53.479861 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b4e98-1e9e-4e23-9d8f-b105a4a4e04f None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:53.480535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b4e98-1e9e-4e23-9d8f-b105a4a4e04f None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:53.481073 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b4e98-1e9e-4e23-9d8f-b105a4a4e04f None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:53.482852 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 384/748] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:53.488660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394b78d-a4b1-4e5c-ba5f-b1e9f67e3217 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:53.489194 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394b78d-a4b1-4e5c-ba5f-b1e9f67e3217 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:53.489652 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394b78d-a4b1-4e5c-ba5f-b1e9f67e3217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:53.499922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 365/749] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:53.512474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-6394b78d-a4b1-4e5c-ba5f-b1e9f67e3217 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:53.513669 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6394b78d-a4b1-4e5c-ba5f-b1e9f67e3217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:53.565719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf04d2-5275-4b5d-8c44-f2f746c7303b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:53.583824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 385/750] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:36:53.679856 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf04d2-5275-4b5d-8c44-f2f746c7303b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:53.697808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:53.717970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd063ba-ab62-4594-aab9-74bbbd79a36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:53.718588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd063ba-ab62-4594-aab9-74bbbd79a36f None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:53.718857 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd063ba-ab62-4594-aab9-74bbbd79a36f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:53.719119 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd063ba-ab62-4594-aab9-74bbbd79a36f None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:53.719807 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd063ba-ab62-4594-aab9-74bbbd79a36f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:53.721221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd063ba-ab62-4594-aab9-74bbbd79a36f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:53.741609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 366/751] 10.222.0.36 () {60 vars in 1141 bytes} [Sat Oct 15 13:36:53 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:36:53.774102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394b78d-a4b1-4e5c-ba5f-b1e9f67e3217 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:53.862957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd063ba-ab62-4594-aab9-74bbbd79a36f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:53.899763 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394b78d-a4b1-4e5c-ba5f-b1e9f67e3217 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:53.902457 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:53.946465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2593e-6294-4bc2-81b9-4ec457cab58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:53.947111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2593e-6294-4bc2-81b9-4ec457cab58b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:53.947394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2593e-6294-4bc2-81b9-4ec457cab58b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:53.947668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2593e-6294-4bc2-81b9-4ec457cab58b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:53.948291 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b2593e-6294-4bc2-81b9-4ec457cab58b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:53.949734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b2593e-6294-4bc2-81b9-4ec457cab58b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-42b2593e-6294-4bc2-81b9-4ec457cab58b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:53.959548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:53.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:53.961508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 386/752] 10.222.0.36 () {56 vars in 1126 bytes} [Sat Oct 15 13:36:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:53.966935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2593e-6294-4bc2-81b9-4ec457cab58b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:54.018011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd063ba-ab62-4594-aab9-74bbbd79a36f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:54.058290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06170c9a-73e5-4ebc-8896-a825c03dd7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:54.059291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06170c9a-73e5-4ebc-8896-a825c03dd7b4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:54.059570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06170c9a-73e5-4ebc-8896-a825c03dd7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:54.059837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06170c9a-73e5-4ebc-8896-a825c03dd7b4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:54.060712 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06170c9a-73e5-4ebc-8896-a825c03dd7b4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:54.062377 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06170c9a-73e5-4ebc-8896-a825c03dd7b4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-06170c9a-73e5-4ebc-8896-a825c03dd7b4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:54.073201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:54.074229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:54.075059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 367/753] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:54.080638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06170c9a-73e5-4ebc-8896-a825c03dd7b4 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:54.139469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:54.139673 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2593e-6294-4bc2-81b9-4ec457cab58b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:54.183596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b408a-5f6d-4c17-9717-8b73fdcdc292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:54.184397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b408a-5f6d-4c17-9717-8b73fdcdc292 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:54.184642 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b408a-5f6d-4c17-9717-8b73fdcdc292 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:54.184916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b408a-5f6d-4c17-9717-8b73fdcdc292 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:54.185562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b408a-5f6d-4c17-9717-8b73fdcdc292 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:54.185868 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b408a-5f6d-4c17-9717-8b73fdcdc292 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:54.187363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b408a-5f6d-4c17-9717-8b73fdcdc292 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:54.193618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 387/754] 10.222.0.36 () {56 vars in 1122 bytes} [Sat Oct 15 13:36:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:36:54.199016 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b408a-5f6d-4c17-9717-8b73fdcdc292 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:54.213526 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06170c9a-73e5-4ebc-8896-a825c03dd7b4 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:54.228577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:54.261920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f7724-cb60-44ec-9e1e-d6af51b3ef77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:54.262798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f7724-cb60-44ec-9e1e-d6af51b3ef77 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:54.263070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f7724-cb60-44ec-9e1e-d6af51b3ef77 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:54.263330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f7724-cb60-44ec-9e1e-d6af51b3ef77 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:54.264227 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f7724-cb60-44ec-9e1e-d6af51b3ef77 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:54.265033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f7724-cb60-44ec-9e1e-d6af51b3ef77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:54.266731 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f7724-cb60-44ec-9e1e-d6af51b3ef77 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:54.274336 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 368/755] 10.222.0.36 () {56 vars in 1112 bytes} [Sat Oct 15 13:36:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:36:54.279559 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f7724-cb60-44ec-9e1e-d6af51b3ef77 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:54.329134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b408a-5f6d-4c17-9717-8b73fdcdc292 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:54.369198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e32fc5-03bc-4f7d-b19c-26d1999a6624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:54.369991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e32fc5-03bc-4f7d-b19c-26d1999a6624 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:54.370363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e32fc5-03bc-4f7d-b19c-26d1999a6624 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:54.370731 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e32fc5-03bc-4f7d-b19c-26d1999a6624 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:54.371471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e32fc5-03bc-4f7d-b19c-26d1999a6624 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:54.371922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e32fc5-03bc-4f7d-b19c-26d1999a6624 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:54.373470 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e32fc5-03bc-4f7d-b19c-26d1999a6624 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:54.380181 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 388/756] 10.222.0.36 () {56 vars in 1122 bytes} [Sat Oct 15 13:36:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:36:54.386272 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e32fc5-03bc-4f7d-b19c-26d1999a6624 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:54.421927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f7724-cb60-44ec-9e1e-d6af51b3ef77 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:54.425724 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:54.464202 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84dce9-1f5a-4deb-95bc-5341d0d29110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:54.465040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84dce9-1f5a-4deb-95bc-5341d0d29110 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:54.465478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84dce9-1f5a-4deb-95bc-5341d0d29110 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:54.465918 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84dce9-1f5a-4deb-95bc-5341d0d29110 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:54.466779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84dce9-1f5a-4deb-95bc-5341d0d29110 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:54.467167 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84dce9-1f5a-4deb-95bc-5341d0d29110 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:54.468798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84dce9-1f5a-4deb-95bc-5341d0d29110 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:54.475515 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 369/757] 10.222.0.36 () {56 vars in 1112 bytes} [Sat Oct 15 13:36:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:36:54.482270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84dce9-1f5a-4deb-95bc-5341d0d29110 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:54.511692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:54.525685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e32fc5-03bc-4f7d-b19c-26d1999a6624 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:54.569031 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b531d-07fe-4769-a5ef-593dbc7151ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:54.570714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b531d-07fe-4769-a5ef-593dbc7151ca None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:54.571064 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b531d-07fe-4769-a5ef-593dbc7151ca None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:54.571399 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b531d-07fe-4769-a5ef-593dbc7151ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:54.572203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b531d-07fe-4769-a5ef-593dbc7151ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:54.573682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b531d-07fe-4769-a5ef-593dbc7151ca None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:54.600061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 389/758] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:36:54 2022] POST /identity/v3/endpoints => generated 325 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:36:54.612556 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84dce9-1f5a-4deb-95bc-5341d0d29110 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:54.662688 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb178b0-1978-4a8f-9a38-07499d662eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:54.663334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb178b0-1978-4a8f-9a38-07499d662eeb None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:54.663611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb178b0-1978-4a8f-9a38-07499d662eeb None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:54.663913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb178b0-1978-4a8f-9a38-07499d662eeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:54.664824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb178b0-1978-4a8f-9a38-07499d662eeb None admin] RBAC: Adding query filter params (interface=public, service_id=d99caee3b2ba4316a78f7fc27e6f91a8, region_id=RegionOne) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:54.665148 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb178b0-1978-4a8f-9a38-07499d662eeb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:54.666843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb178b0-1978-4a8f-9a38-07499d662eeb None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:54.674012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 370/759] 10.222.0.36 () {56 vars in 1251 bytes} [Sat Oct 15 13:36:54 2022] GET /identity/v3/endpoints?service_id=d99caee3b2ba4316a78f7fc27e6f91a8&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:36:54.679915 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b531d-07fe-4769-a5ef-593dbc7151ca None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:54.774373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51d441-3c26-43c3-8682-5baf33e132bc None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:54.774718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51d441-3c26-43c3-8682-5baf33e132bc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:54.775186 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51d441-3c26-43c3-8682-5baf33e132bc None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:54.776723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 371/760] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:36:54.785183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeaa550-523d-49b8-8081-2b2c88fd3a8e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:54.785617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeaa550-523d-49b8-8081-2b2c88fd3a8e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:54.785987 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeaa550-523d-49b8-8081-2b2c88fd3a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:54.803947 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b531d-07fe-4769-a5ef-593dbc7151ca None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:54.806779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-eaeaa550-523d-49b8-8081-2b2c88fd3a8e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:54.808132 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-eaeaa550-523d-49b8-8081-2b2c88fd3a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:54.838518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:54.850919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d765c-5630-489e-b91e-d540406f79ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:54.851554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d765c-5630-489e-b91e-d540406f79ab None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:54.851834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d765c-5630-489e-b91e-d540406f79ab None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:54.852232 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d765c-5630-489e-b91e-d540406f79ab None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:54.852847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d765c-5630-489e-b91e-d540406f79ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:54.854216 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d765c-5630-489e-b91e-d540406f79ab None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:54.863779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 390/761] 10.222.0.36 () {56 vars in 1087 bytes} [Sat Oct 15 13:36:54 2022] GET /identity/v3/services => generated 1957 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:36:54.897849 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 372/762] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:54 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:55.165653 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d765c-5630-489e-b91e-d540406f79ab None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:55.182792 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:55.278634 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d765c-5630-489e-b91e-d540406f79ab None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:55.313663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29fab0-59e6-4859-af06-8d8cacd3d953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:55.314535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29fab0-59e6-4859-af06-8d8cacd3d953 None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:55.314992 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29fab0-59e6-4859-af06-8d8cacd3d953 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:55.315409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29fab0-59e6-4859-af06-8d8cacd3d953 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:55.316330 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29fab0-59e6-4859-af06-8d8cacd3d953 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:55.317972 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29fab0-59e6-4859-af06-8d8cacd3d953 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:55.320716 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: WARNING keystone.server.flask.common [None req-1a29fab0-59e6-4859-af06-8d8cacd3d953 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 15 13:36:55.334863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.sql.core [None req-1a29fab0-59e6-4859-af06-8d8cacd3d953 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 15 13:36:55.334863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 15 13:36:55.334863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [parameters: {'user_id': '4d7034eb148f45f7b39f64ffc486fef9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 15 13:36:55.334863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87915) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-1a29fab0-59e6-4859-af06-8d8cacd3d953 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application conn.query(q) Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application result.read() Oct 15 13:36:55.342998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 13:36:55.344167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application next(self.gen) Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application yield resource Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application next(self.gen) Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise self.value Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application session.commit() Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 13:36:55.345370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self.session.flush() Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application compat.raise_( Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 13:36:55.346458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 13:36:55.347397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application conn.query(q) Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application result.read() Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 13:36:55.348317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [parameters: {'user_id': '4d7034eb148f45f7b39f64ffc486fef9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:55.349169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 15 13:36:55.349942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 15 13:36:55.350818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 15 13:36:55.350818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 15 13:36:55.350818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 13:36:55.350818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 13:36:55.350818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 15 13:36:55.350818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:36:55.352822 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 391/763] 10.222.0.36 () {60 vars in 1131 bytes} [Sat Oct 15 13:36:55 2022] POST /identity/v3/users => generated 160 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:55.359529 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeaa550-523d-49b8-8081-2b2c88fd3a8e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:55.462274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeaa550-523d-49b8-8081-2b2c88fd3a8e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:55.503101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d049c6-c835-46b5-9de6-357089c8bcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:55.503747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d049c6-c835-46b5-9de6-357089c8bcb9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:55.504047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d049c6-c835-46b5-9de6-357089c8bcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:55.504310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d049c6-c835-46b5-9de6-357089c8bcb9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:55.512145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d049c6-c835-46b5-9de6-357089c8bcb9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:55.513611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d049c6-c835-46b5-9de6-357089c8bcb9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-a7d049c6-c835-46b5-9de6-357089c8bcb9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:36:55.525923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:55.526914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:55.527845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:55.527845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:55.527845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:36:55.527845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:36:55.527845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:36:55.527845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:36:55.527845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:36:55.527845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:36:55.527845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 15 13:36:55.527845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:55.527845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 373/764] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:36:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:55.535723 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29fab0-59e6-4859-af06-8d8cacd3d953 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:55.619616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:55.644743 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29fab0-59e6-4859-af06-8d8cacd3d953 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:55.682770 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb248d54-ec05-48ff-bd9b-47ab417deb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:55.683793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb248d54-ec05-48ff-bd9b-47ab417deb3c None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:55.684317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb248d54-ec05-48ff-bd9b-47ab417deb3c None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:55.684782 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb248d54-ec05-48ff-bd9b-47ab417deb3c None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:55.685685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb248d54-ec05-48ff-bd9b-47ab417deb3c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:55.686206 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb248d54-ec05-48ff-bd9b-47ab417deb3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:55.688063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb248d54-ec05-48ff-bd9b-47ab417deb3c None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:55.727628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 392/765] 10.222.0.36 () {56 vars in 1101 bytes} [Sat Oct 15 13:36:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:56.053468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48290811-892e-4ec4-878e-4e66dfce5990 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:56.053667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48290811-892e-4ec4-878e-4e66dfce5990 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:56.053909 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48290811-892e-4ec4-878e-4e66dfce5990 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:56.055305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 374/766] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:56.060684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6bf91-530f-4c19-ad59-e715aeaea50b None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:56.060972 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6bf91-530f-4c19-ad59-e715aeaea50b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:56.061241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6bf91-530f-4c19-ad59-e715aeaea50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:56.080736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-f3d6bf91-530f-4c19-ad59-e715aeaea50b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:56.082237 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f3d6bf91-530f-4c19-ad59-e715aeaea50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:56.154318 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 393/767] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:56 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:56.419627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48290811-892e-4ec4-878e-4e66dfce5990 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:56.441535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:56.532733 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48290811-892e-4ec4-878e-4e66dfce5990 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:56.571030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b3219-22de-407b-8140-6f47cfb65568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:56.571862 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b3219-22de-407b-8140-6f47cfb65568 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:56.572463 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b3219-22de-407b-8140-6f47cfb65568 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:56.572463 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b3219-22de-407b-8140-6f47cfb65568 None admin] PATH_INFO: `/v3/services/image` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:56.572802 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b3219-22de-407b-8140-6f47cfb65568 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:56.574284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b3219-22de-407b-8140-6f47cfb65568 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-840b3219-22de-407b-8140-6f47cfb65568 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:56.584713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:56.585785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:56.586669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 375/768] 10.222.0.36 () {56 vars in 1105 bytes} [Sat Oct 15 13:36:56 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:56.592392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6bf91-530f-4c19-ad59-e715aeaea50b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:56.780631 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:56.822291 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6bf91-530f-4c19-ad59-e715aeaea50b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:56.862259 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2cdba-5800-41b7-8023-3d78028f574d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:56.862884 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2cdba-5800-41b7-8023-3d78028f574d None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:56.863155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2cdba-5800-41b7-8023-3d78028f574d None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:56.863420 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2cdba-5800-41b7-8023-3d78028f574d None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:56.864093 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2cdba-5800-41b7-8023-3d78028f574d None admin] RBAC: Adding query filter params (name=image) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:56.864398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2cdba-5800-41b7-8023-3d78028f574d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:56.865794 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2cdba-5800-41b7-8023-3d78028f574d None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:56.871677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 394/769] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:56.877985 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b3219-22de-407b-8140-6f47cfb65568 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:56.957959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db8696-0093-4086-8869-11d35e1cf108 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:56.958227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db8696-0093-4086-8869-11d35e1cf108 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:56.958487 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db8696-0093-4086-8869-11d35e1cf108 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:56.959936 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 395/770] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:36:56.966157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed171648-2e22-48e1-9c11-8f2bba70f2d8 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:56.966424 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed171648-2e22-48e1-9c11-8f2bba70f2d8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:56.966695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed171648-2e22-48e1-9c11-8f2bba70f2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:56.976361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:56.988171 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-ed171648-2e22-48e1-9c11-8f2bba70f2d8 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:56.989396 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ed171648-2e22-48e1-9c11-8f2bba70f2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:56.997039 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b3219-22de-407b-8140-6f47cfb65568 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:57.033976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198dfb2-9af3-47c1-ba22-5325fa56c0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:57.034630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198dfb2-9af3-47c1-ba22-5325fa56c0ac None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:57.034804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198dfb2-9af3-47c1-ba22-5325fa56c0ac None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:57.035047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198dfb2-9af3-47c1-ba22-5325fa56c0ac None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:57.035671 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198dfb2-9af3-47c1-ba22-5325fa56c0ac None admin] RBAC: Adding query filter params (type=image) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:57.036000 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198dfb2-9af3-47c1-ba22-5325fa56c0ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:57.037382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198dfb2-9af3-47c1-ba22-5325fa56c0ac None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:57.042445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 376/771] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:57.048466 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198dfb2-9af3-47c1-ba22-5325fa56c0ac None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:57.084458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 396/772] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:56 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:36:57.112294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94516c2-18dd-4c56-aa46-d9c9e57cdb18 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:57.112575 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94516c2-18dd-4c56-aa46-d9c9e57cdb18 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:57.112845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94516c2-18dd-4c56-aa46-d9c9e57cdb18 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:57.114241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 397/773] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:36:57.119617 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd6401-a906-40ba-80d2-b44627a899b4 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:57.119913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd6401-a906-40ba-80d2-b44627a899b4 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:57.120210 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd6401-a906-40ba-80d2-b44627a899b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:57.139977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-4dbd6401-a906-40ba-80d2-b44627a899b4 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:57.140954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4dbd6401-a906-40ba-80d2-b44627a899b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:57.158764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198dfb2-9af3-47c1-ba22-5325fa56c0ac None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:57.196329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909157df-7363-4473-a922-938dff2ccb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:57.196946 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909157df-7363-4473-a922-938dff2ccb2b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:57.197219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909157df-7363-4473-a922-938dff2ccb2b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:57.197479 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909157df-7363-4473-a922-938dff2ccb2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:57.197951 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:57.199000 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909157df-7363-4473-a922-938dff2ccb2b None admin] RBAC: Adding query filter params (interface=public, service_id=230399b9759847339d164ac5232f2bbe, region_id=RegionOne) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:57.199610 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909157df-7363-4473-a922-938dff2ccb2b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:57.201588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909157df-7363-4473-a922-938dff2ccb2b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:57.207775 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 377/774] 10.222.0.36 () {56 vars in 1251 bytes} [Sat Oct 15 13:36:57 2022] GET /identity/v3/endpoints?service_id=230399b9759847339d164ac5232f2bbe&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:36:57.212854 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 398/775] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:57 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:36:57.213954 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909157df-7363-4473-a922-938dff2ccb2b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:57.226970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:57.322883 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909157df-7363-4473-a922-938dff2ccb2b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:57.341786 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd6401-a906-40ba-80d2-b44627a899b4 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:57.361757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21507489-db41-4307-93eb-2767db2b5481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:57.362627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21507489-db41-4307-93eb-2767db2b5481 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:57.362738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21507489-db41-4307-93eb-2767db2b5481 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:57.363076 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21507489-db41-4307-93eb-2767db2b5481 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:57.363801 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21507489-db41-4307-93eb-2767db2b5481 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:57.365536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21507489-db41-4307-93eb-2767db2b5481 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:57.373283 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 378/776] 10.222.0.36 () {56 vars in 1087 bytes} [Sat Oct 15 13:36:57 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:36:57.467195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd6401-a906-40ba-80d2-b44627a899b4 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:57.487321 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21507489-db41-4307-93eb-2767db2b5481 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:57.510562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca148d1-820f-4531-906f-4819d10cede7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:57.511232 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca148d1-820f-4531-906f-4819d10cede7 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:57.511510 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca148d1-820f-4531-906f-4819d10cede7 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:57.511781 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca148d1-820f-4531-906f-4819d10cede7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:57.513096 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca148d1-820f-4531-906f-4819d10cede7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:57.514567 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca148d1-820f-4531-906f-4819d10cede7 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:57.516341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 399/777] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:36:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:57.522422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca148d1-820f-4531-906f-4819d10cede7 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:57.588728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:57.596270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:57.621816 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21507489-db41-4307-93eb-2767db2b5481 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:57.640757 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca148d1-820f-4531-906f-4819d10cede7 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:57.660110 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ece3a-cbba-426f-a672-a206ef97ee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:57.660752 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ece3a-cbba-426f-a672-a206ef97ee1b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:57.661021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ece3a-cbba-426f-a672-a206ef97ee1b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:57.661285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ece3a-cbba-426f-a672-a206ef97ee1b None admin] PATH_INFO: `/v3/services/compute` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:57.661859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ece3a-cbba-426f-a672-a206ef97ee1b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:57.663254 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ece3a-cbba-426f-a672-a206ef97ee1b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f01ece3a-cbba-426f-a672-a206ef97ee1b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:36:57.672287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:36:57.673602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:36:57.674803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 379/778] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:36:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:36:57.680301 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ece3a-cbba-426f-a672-a206ef97ee1b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:57.682737 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddace81c-3385-4fc8-90c7-0626f55ba18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:57.683652 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddace81c-3385-4fc8-90c7-0626f55ba18c None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:57.684197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddace81c-3385-4fc8-90c7-0626f55ba18c None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:57.684490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddace81c-3385-4fc8-90c7-0626f55ba18c None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:57.685149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddace81c-3385-4fc8-90c7-0626f55ba18c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:57.686594 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddace81c-3385-4fc8-90c7-0626f55ba18c None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:57.727993 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 400/779] 10.222.0.36 () {60 vars in 1141 bytes} [Sat Oct 15 13:36:57 2022] POST /identity/v3/projects => generated 311 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:36:57.801673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ece3a-cbba-426f-a672-a206ef97ee1b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:57.840878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb6338-5a54-4a17-87eb-54ab71395b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:57.841438 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb6338-5a54-4a17-87eb-54ab71395b61 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:57.841710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb6338-5a54-4a17-87eb-54ab71395b61 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:57.841973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb6338-5a54-4a17-87eb-54ab71395b61 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:57.842655 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb6338-5a54-4a17-87eb-54ab71395b61 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:57.842951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb6338-5a54-4a17-87eb-54ab71395b61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:57.844394 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb6338-5a54-4a17-87eb-54ab71395b61 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:57.850412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 380/780] 10.222.0.36 () {56 vars in 1112 bytes} [Sat Oct 15 13:36:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:36:57.856171 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddace81c-3385-4fc8-90c7-0626f55ba18c None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:57.995337 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddace81c-3385-4fc8-90c7-0626f55ba18c None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:58.006500 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:58.035544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07b93b-d832-48ba-8cc3-2e498860046f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:58.036200 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07b93b-d832-48ba-8cc3-2e498860046f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:58.036481 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07b93b-d832-48ba-8cc3-2e498860046f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:58.036748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07b93b-d832-48ba-8cc3-2e498860046f None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:58.037389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07b93b-d832-48ba-8cc3-2e498860046f None admin] RBAC: Adding query filter params (type=compute) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:36:58.037686 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07b93b-d832-48ba-8cc3-2e498860046f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:58.039072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07b93b-d832-48ba-8cc3-2e498860046f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:58.049656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 401/781] 10.222.0.36 () {56 vars in 1112 bytes} [Sat Oct 15 13:36:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:36:58.052321 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb6338-5a54-4a17-87eb-54ab71395b61 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:36:58.055286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:58.164383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb6338-5a54-4a17-87eb-54ab71395b61 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:36:58.199713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdde69-2a54-49c4-bef7-948c2f5bb48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:36:58.200361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdde69-2a54-49c4-bef7-948c2f5bb48e None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:58.200640 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdde69-2a54-49c4-bef7-948c2f5bb48e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:58.200907 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdde69-2a54-49c4-bef7-948c2f5bb48e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:58.201569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bdde69-2a54-49c4-bef7-948c2f5bb48e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:36:58.202957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bdde69-2a54-49c4-bef7-948c2f5bb48e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:36:58.235904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 381/782] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:36:58 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:36:59.596894 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5f2d3-e8d4-4ff5-b1a9-1feb40c94053 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:59.597449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5f2d3-e8d4-4ff5-b1a9-1feb40c94053 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:59.597449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5f2d3-e8d4-4ff5-b1a9-1feb40c94053 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:59.599042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 402/783] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:59.604738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001ada-68cd-49ba-803a-e6baa1f69287 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:59.605219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001ada-68cd-49ba-803a-e6baa1f69287 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:59.605452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001ada-68cd-49ba-803a-e6baa1f69287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:59.625354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-08001ada-68cd-49ba-803a-e6baa1f69287 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:59.626592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-08001ada-68cd-49ba-803a-e6baa1f69287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:59.683716 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:36:59.707013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 382/784] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:36:59.944767 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4f662-70b8-4300-b612-6aa074b03eed None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:59.945147 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4f662-70b8-4300-b612-6aa074b03eed None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:59.945263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4f662-70b8-4300-b612-6aa074b03eed None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:59.946598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 403/785] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:36:59.952203 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b577fe5-cf54-4282-b099-d193d255031d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:36:59.952544 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b577fe5-cf54-4282-b099-d193d255031d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:36:59.952871 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b577fe5-cf54-4282-b099-d193d255031d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:36:59.971950 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2b577fe5-cf54-4282-b099-d193d255031d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:36:59.973148 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2b577fe5-cf54-4282-b099-d193d255031d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:36:59.973684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4f662-70b8-4300-b612-6aa074b03eed None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:00.051976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 383/786] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:36:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:00.087209 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4f662-70b8-4300-b612-6aa074b03eed None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:00.124144 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b75338-28ab-4ca8-8a17-0e1a0889bc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:00.124769 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b75338-28ab-4ca8-8a17-0e1a0889bc29 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:00.125016 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b75338-28ab-4ca8-8a17-0e1a0889bc29 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:00.125276 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b75338-28ab-4ca8-8a17-0e1a0889bc29 None admin] PATH_INFO: `/v3/services/image` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:00.125956 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b75338-28ab-4ca8-8a17-0e1a0889bc29 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:00.127508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b75338-28ab-4ca8-8a17-0e1a0889bc29 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-f3b75338-28ab-4ca8-8a17-0e1a0889bc29 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:00.136867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:00.137886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:37:00.138911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 404/787] 10.222.0.36 () {56 vars in 1105 bytes} [Sat Oct 15 13:36:59 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:37:00.145117 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b577fe5-cf54-4282-b099-d193d255031d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:00.251823 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b577fe5-cf54-4282-b099-d193d255031d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:00.272744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:00.305952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e46ab5-5d14-4ce4-aa39-2fb621527481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:00.306929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e46ab5-5d14-4ce4-aa39-2fb621527481 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:00.307390 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e46ab5-5d14-4ce4-aa39-2fb621527481 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:00.307850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e46ab5-5d14-4ce4-aa39-2fb621527481 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:00.308951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e46ab5-5d14-4ce4-aa39-2fb621527481 None admin] RBAC: Adding query filter params (name=image) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:00.309464 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e46ab5-5d14-4ce4-aa39-2fb621527481 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:00.311841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e46ab5-5d14-4ce4-aa39-2fb621527481 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:00.325638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 384/788] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:37:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:00.327083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b75338-28ab-4ca8-8a17-0e1a0889bc29 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:00.338250 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e46ab5-5d14-4ce4-aa39-2fb621527481 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:00.378871 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:00.450361 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b75338-28ab-4ca8-8a17-0e1a0889bc29 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:00.453259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e46ab5-5d14-4ce4-aa39-2fb621527481 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:00.487007 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219f38d-43d2-4827-b6d8-36fdfab4e123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:00.487609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219f38d-43d2-4827-b6d8-36fdfab4e123 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:00.487899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219f38d-43d2-4827-b6d8-36fdfab4e123 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:00.488174 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219f38d-43d2-4827-b6d8-36fdfab4e123 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:00.488799 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219f38d-43d2-4827-b6d8-36fdfab4e123 None admin] RBAC: Adding query filter params (type=image) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:00.489095 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219f38d-43d2-4827-b6d8-36fdfab4e123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:00.489518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8e867-2783-48b8-be2b-f778600deebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:00.490155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8e867-2783-48b8-be2b-f778600deebb None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:00.490428 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8e867-2783-48b8-be2b-f778600deebb None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:00.490515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219f38d-43d2-4827-b6d8-36fdfab4e123 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:00.490697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8e867-2783-48b8-be2b-f778600deebb None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:00.491794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8e867-2783-48b8-be2b-f778600deebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:00.493250 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8e867-2783-48b8-be2b-f778600deebb None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:00.494823 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 385/789] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:37:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:37:00.495512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 405/790] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:37:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:00.495981 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a6cb3-2632-4756-bec1-caca300776a0 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:00.496269 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a6cb3-2632-4756-bec1-caca300776a0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:00.496576 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a6cb3-2632-4756-bec1-caca300776a0 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:00.499905 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 386/791] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:00.502667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a6cb3-2632-4756-bec1-caca300776a0 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:00.502800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219f38d-43d2-4827-b6d8-36fdfab4e123 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:00.551161 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:00.602524 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:00.606671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219f38d-43d2-4827-b6d8-36fdfab4e123 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:00.614733 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a6cb3-2632-4756-bec1-caca300776a0 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:00.643415 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cab4b-3077-4d03-a614-f44702817506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:00.644070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cab4b-3077-4d03-a614-f44702817506 None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:00.644348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cab4b-3077-4d03-a614-f44702817506 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:00.644624 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cab4b-3077-4d03-a614-f44702817506 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:00.645299 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cab4b-3077-4d03-a614-f44702817506 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:00.646658 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cab4b-3077-4d03-a614-f44702817506 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:00.649489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07abcb9-f09a-412e-87dc-6297b6f15e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:00.650104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07abcb9-f09a-412e-87dc-6297b6f15e00 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:00.650379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07abcb9-f09a-412e-87dc-6297b6f15e00 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:00.650645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07abcb9-f09a-412e-87dc-6297b6f15e00 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:00.651289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07abcb9-f09a-412e-87dc-6297b6f15e00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:00.652755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07abcb9-f09a-412e-87dc-6297b6f15e00 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:00.669203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 406/792] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:37:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:37:00.670558 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04dbe3-0347-4efe-8df6-95ff75844e9a None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:00.670826 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04dbe3-0347-4efe-8df6-95ff75844e9a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:00.671099 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04dbe3-0347-4efe-8df6-95ff75844e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:00.687076 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 387/793] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:37:00 2022] POST /identity/v3/users => generated 285 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:37:00.697994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-9b04dbe3-0347-4efe-8df6-95ff75844e9a None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:00.699390 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9b04dbe3-0347-4efe-8df6-95ff75844e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:00.804003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 407/794] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:01.068694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07abcb9-f09a-412e-87dc-6297b6f15e00 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:01.189098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07abcb9-f09a-412e-87dc-6297b6f15e00 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:01.191163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:01.227104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209f5ce-e512-44c7-96b8-61a617355ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:01.227781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209f5ce-e512-44c7-96b8-61a617355ed1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:01.228086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209f5ce-e512-44c7-96b8-61a617355ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:01.228357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209f5ce-e512-44c7-96b8-61a617355ed1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:01.240188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209f5ce-e512-44c7-96b8-61a617355ed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-c209f5ce-e512-44c7-96b8-61a617355ed1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:37:01.245028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:01.245951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:01.246757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:01.246757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:01.246757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:01.246757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:01.246757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:37:01.246757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:37:01.246757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:37:01.246757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:37:01.246757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:37:01.246757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:37:01.246757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 13:37:01.246757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:01.246757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 388/795] 10.222.0.36 () {56 vars in 1120 bytes} [Sat Oct 15 13:37:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:37:01.252560 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04dbe3-0347-4efe-8df6-95ff75844e9a None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:01.336622 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:01.356212 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04dbe3-0347-4efe-8df6-95ff75844e9a None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:01.390798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d534f-9b77-439e-92f9-eede24ea64e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:01.391418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d534f-9b77-439e-92f9-eede24ea64e3 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:01.391801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d534f-9b77-439e-92f9-eede24ea64e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:01.392104 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d534f-9b77-439e-92f9-eede24ea64e3 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:01.392779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d534f-9b77-439e-92f9-eede24ea64e3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:01.393062 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d534f-9b77-439e-92f9-eede24ea64e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:01.394450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d534f-9b77-439e-92f9-eede24ea64e3 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:01.403383 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 408/796] 10.222.0.36 () {56 vars in 1115 bytes} [Sat Oct 15 13:37:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:01.409103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209f5ce-e512-44c7-96b8-61a617355ed1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:01.518132 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209f5ce-e512-44c7-96b8-61a617355ed1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:01.552885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f603d-9a0e-4432-88c2-788e35c3fdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:01.553538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f603d-9a0e-4432-88c2-788e35c3fdc7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:01.553823 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f603d-9a0e-4432-88c2-788e35c3fdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:01.554076 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f603d-9a0e-4432-88c2-788e35c3fdc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:01.564894 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f603d-9a0e-4432-88c2-788e35c3fdc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:01.566371 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f603d-9a0e-4432-88c2-788e35c3fdc7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-e70f603d-9a0e-4432-88c2-788e35c3fdc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:37:01.580505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:01.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:37:01.587722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:37:01.587722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:37:01.587722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:01.587722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 389/797] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:37:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:37:01.587991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d534f-9b77-439e-92f9-eede24ea64e3 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:01.696055 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d534f-9b77-439e-92f9-eede24ea64e3 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:01.730129 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6630997-1495-4ecf-ade6-1277f21c0b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:01.730735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6630997-1495-4ecf-ade6-1277f21c0b6a None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:01.731008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6630997-1495-4ecf-ade6-1277f21c0b6a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:01.731274 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6630997-1495-4ecf-ade6-1277f21c0b6a None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:01.731916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6630997-1495-4ecf-ade6-1277f21c0b6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:01.732231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6630997-1495-4ecf-ade6-1277f21c0b6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:01.733581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6630997-1495-4ecf-ade6-1277f21c0b6a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:01.746304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 409/798] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:01.752661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f603d-9a0e-4432-88c2-788e35c3fdc7 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:01.768928 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:01.855971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f603d-9a0e-4432-88c2-788e35c3fdc7 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:01.897839 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144edd7-814d-4ad8-a991-a96255991edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:01.898479 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144edd7-814d-4ad8-a991-a96255991edc None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:01.898753 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144edd7-814d-4ad8-a991-a96255991edc None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:01.899018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144edd7-814d-4ad8-a991-a96255991edc None admin] PATH_INFO: `/v3/users/nova` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:01.907765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144edd7-814d-4ad8-a991-a96255991edc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:01.909268 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144edd7-814d-4ad8-a991-a96255991edc None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-a144edd7-814d-4ad8-a991-a96255991edc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:37:01.920709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:01.921663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:01.922595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:01.922595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:01.922595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:37:01.922595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:37:01.922595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:37:01.922595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:37:01.922595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:37:01.922595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:37:01.922595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 13:37:01.922595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:01.926842 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 390/799] 10.222.0.36 () {56 vars in 1093 bytes} [Sat Oct 15 13:37:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:37:01.927691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6630997-1495-4ecf-ade6-1277f21c0b6a None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:01.931804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:02.029769 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6630997-1495-4ecf-ade6-1277f21c0b6a None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:02.064175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55d018-5eb6-4c49-bb7b-291b22dd9536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:02.064795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55d018-5eb6-4c49-bb7b-291b22dd9536 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:02.065072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55d018-5eb6-4c49-bb7b-291b22dd9536 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:02.065339 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55d018-5eb6-4c49-bb7b-291b22dd9536 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:02.066065 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55d018-5eb6-4c49-bb7b-291b22dd9536 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:02.066369 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55d018-5eb6-4c49-bb7b-291b22dd9536 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:02.067790 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55d018-5eb6-4c49-bb7b-291b22dd9536 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:02.085734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 410/800] 10.222.0.36 () {56 vars in 1133 bytes} [Sat Oct 15 13:37:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:02.091450 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144edd7-814d-4ad8-a991-a96255991edc None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:02.173119 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:02.198911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144edd7-814d-4ad8-a991-a96255991edc None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:02.232311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deac5d-37bf-46ff-b143-7f3c48e5e1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:02.232940 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deac5d-37bf-46ff-b143-7f3c48e5e1dd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:02.233212 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deac5d-37bf-46ff-b143-7f3c48e5e1dd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:02.233476 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deac5d-37bf-46ff-b143-7f3c48e5e1dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:02.244489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02deac5d-37bf-46ff-b143-7f3c48e5e1dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:02.245941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02deac5d-37bf-46ff-b143-7f3c48e5e1dd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-02deac5d-37bf-46ff-b143-7f3c48e5e1dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:37:02.260036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:02.261002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:37:02.261888 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:37:02.261888 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:37:02.261888 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:02.262692 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 391/801] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:37:02.266904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55d018-5eb6-4c49-bb7b-291b22dd9536 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:02.330651 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:02.371522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55d018-5eb6-4c49-bb7b-291b22dd9536 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:02.408175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d3a44-168c-4894-ae12-9c9c01082c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:02.408788 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d3a44-168c-4894-ae12-9c9c01082c1f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:02.409065 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d3a44-168c-4894-ae12-9c9c01082c1f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:02.409340 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d3a44-168c-4894-ae12-9c9c01082c1f None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:02.409984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d3a44-168c-4894-ae12-9c9c01082c1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:02.410289 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d3a44-168c-4894-ae12-9c9c01082c1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:02.411659 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d3a44-168c-4894-ae12-9c9c01082c1f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:02.423519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 411/802] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:02.429218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deac5d-37bf-46ff-b143-7f3c48e5e1dd None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:02.532234 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deac5d-37bf-46ff-b143-7f3c48e5e1dd None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:02.558300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:02.568512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f3d20-5a63-407d-a5a5-c84e5df7acc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:02.569149 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f3d20-5a63-407d-a5a5-c84e5df7acc1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:02.569423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f3d20-5a63-407d-a5a5-c84e5df7acc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:02.569697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f3d20-5a63-407d-a5a5-c84e5df7acc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:02.576715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f3d20-5a63-407d-a5a5-c84e5df7acc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:02.578167 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f3d20-5a63-407d-a5a5-c84e5df7acc1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-375f3d20-5a63-407d-a5a5-c84e5df7acc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:37:02.589970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:02.590897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:02.591760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:37:02.591760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:37:02.591760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:37:02.591760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:37:02.591760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:37:02.591760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:37:02.591760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:37:02.591760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:02.595991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 392/803] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:37:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:37:02.598418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d3a44-168c-4894-ae12-9c9c01082c1f None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:02.701567 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d3a44-168c-4894-ae12-9c9c01082c1f None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:02.724804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:02.738399 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9918ab5-8528-4b1c-b30c-88257216077d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:02.739046 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9918ab5-8528-4b1c-b30c-88257216077d None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:02.739324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9918ab5-8528-4b1c-b30c-88257216077d None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:02.739593 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9918ab5-8528-4b1c-b30c-88257216077d None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:02.740304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9918ab5-8528-4b1c-b30c-88257216077d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:02.740611 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9918ab5-8528-4b1c-b30c-88257216077d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:02.742035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9918ab5-8528-4b1c-b30c-88257216077d None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:02.755782 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 412/804] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:37:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:02.761916 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f3d20-5a63-407d-a5a5-c84e5df7acc1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:02.879650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f3d20-5a63-407d-a5a5-c84e5df7acc1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:02.908145 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14417640-732b-465a-bbdb-10ebb45153e6 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:02.908500 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14417640-732b-465a-bbdb-10ebb45153e6 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:02.908693 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14417640-732b-465a-bbdb-10ebb45153e6 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:02.910193 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 413/805] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:37:02.915559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5fe14-91e3-4a59-9e2f-da12fee909f1 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:02.915834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5fe14-91e3-4a59-9e2f-da12fee909f1 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:02.916121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5fe14-91e3-4a59-9e2f-da12fee909f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:02.919110 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4feb456-8d81-4dfd-8c7a-94de9b18fded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:02.919765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4feb456-8d81-4dfd-8c7a-94de9b18fded None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:02.920071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4feb456-8d81-4dfd-8c7a-94de9b18fded None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:02.920341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4feb456-8d81-4dfd-8c7a-94de9b18fded None admin] PATH_INFO: `/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/2e13692c1e83413a909d99bb7db2e4b8/roles/5bd4f51ce7594d178e68f303bb34cced` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:02.922742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4feb456-8d81-4dfd-8c7a-94de9b18fded None admin] RBAC: Authorizing `identity:create_grant(project_id=2a5a64cc8aa04972bb464407cb0312b4, user_id=2e13692c1e83413a909d99bb7db2e4b8, role_id=5bd4f51ce7594d178e68f303bb34cced)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:02.925366 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4feb456-8d81-4dfd-8c7a-94de9b18fded None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:02.935331 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-72b5fe14-91e3-4a59-9e2f-da12fee909f1 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:02.936759 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-72b5fe14-91e3-4a59-9e2f-da12fee909f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:02.940273 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 393/806] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:37:02 2022] PUT /identity/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/2e13692c1e83413a909d99bb7db2e4b8/roles/5bd4f51ce7594d178e68f303bb34cced => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:37:02.941621 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65ba61-d6c7-4cf5-abe5-90756a5569f2 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:02.941910 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65ba61-d6c7-4cf5-abe5-90756a5569f2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:02.942180 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65ba61-d6c7-4cf5-abe5-90756a5569f2 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:02.943443 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 394/807] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:02.949638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c81b2-2ed7-488e-96f3-1433636d0216 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:02.949915 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c81b2-2ed7-488e-96f3-1433636d0216 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:02.950185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c81b2-2ed7-488e-96f3-1433636d0216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:02.970584 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a39c81b2-2ed7-488e-96f3-1433636d0216 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:02.971829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a39c81b2-2ed7-488e-96f3-1433636d0216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:03.023701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 414/808] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:03.024609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:03.053276 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 395/809] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:37:03.285041 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5fe14-91e3-4a59-9e2f-da12fee909f1 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:03.318542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c81b2-2ed7-488e-96f3-1433636d0216 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:03.388231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5fe14-91e3-4a59-9e2f-da12fee909f1 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:03.402503 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:03.426234 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1114f-3ef2-4dee-bc4e-b9cbdee2a94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:03.426883 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1114f-3ef2-4dee-bc4e-b9cbdee2a94b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:03.427160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1114f-3ef2-4dee-bc4e-b9cbdee2a94b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:03.427430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1114f-3ef2-4dee-bc4e-b9cbdee2a94b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:03.429253 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c81b2-2ed7-488e-96f3-1433636d0216 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:03.439314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1114f-3ef2-4dee-bc4e-b9cbdee2a94b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-0db1114f-3ef2-4dee-bc4e-b9cbdee2a94b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:37:03.444172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:03.445137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:03.445987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:03.445987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:03.445987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:03.445987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:03.445987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:37:03.445987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:37:03.445987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:37:03.445987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:37:03.445987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:37:03.445987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:37:03.445987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:37:03.445987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:37:03.445987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 415/810] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:37:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:37:03.451729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1114f-3ef2-4dee-bc4e-b9cbdee2a94b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:03.468560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e79b94-f4a6-4723-84b2-ef1f22b4c616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:03.469248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e79b94-f4a6-4723-84b2-ef1f22b4c616 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:03.469515 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e79b94-f4a6-4723-84b2-ef1f22b4c616 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:03.469853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e79b94-f4a6-4723-84b2-ef1f22b4c616 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:03.470493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e79b94-f4a6-4723-84b2-ef1f22b4c616 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:03.472045 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e79b94-f4a6-4723-84b2-ef1f22b4c616 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:03.480864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.sql.core [None req-f6e79b94-f4a6-4723-84b2-ef1f22b4c616 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 13:37:03.480864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 13:37:03.480864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [parameters: {'id': 'ed2fc47eb39e4197beffb7b791c41981', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 13:37:03.480864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87916) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 15 13:37:03.489284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f6e79b94-f4a6-4723-84b2-ef1f22b4c616 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 13:37:03.489284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:03.489284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 13:37:03.489284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 13:37:03.489284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 13:37:03.489284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 13:37:03.489284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 13:37:03.489284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 13:37:03.489284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 13:37:03.489284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application conn.query(q) Oct 15 13:37:03.489284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 13:37:03.489284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 13:37:03.489284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 13:37:03.489284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result.read() Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 15 13:37:03.490138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application next(self.gen) Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application yield resource Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application next(self.gen) Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.session.rollback() Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.force_reraise() Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise self.value Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application session.commit() Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 15 13:37:03.491078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._prepare_impl() Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.session.flush() Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._flush(objects) Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application compat.raise_( Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application flush_context.execute() Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rec.execute(self) Oct 15 13:37:03.493401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 15 13:37:03.495160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result = self._query(query) Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application conn.query(q) Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result.read() Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 15 13:37:03.496863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [parameters: {'id': 'ed2fc47eb39e4197beffb7b791c41981', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 15 13:37:03.498074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:03.499858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:03.501913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 15 13:37:03.501913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 15 13:37:03.501913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:03.501913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:03.501913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 15 13:37:03.501913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 15 13:37:03.501913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:03.501913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:03.501913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 15 13:37:03.501913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 15 13:37:03.501913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 15 13:37:03.501913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:03.501913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 396/811] 10.222.0.36 () {60 vars in 1137 bytes} [Sat Oct 15 13:37:03 2022] POST /identity/v3/domains => generated 190 bytes in 174 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:37:03.501913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e79b94-f4a6-4723-84b2-ef1f22b4c616 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:03.550012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:03.569356 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1114f-3ef2-4dee-bc4e-b9cbdee2a94b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:03.607635 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864681a-884d-4ce8-b017-daa6415037a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:03.608472 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864681a-884d-4ce8-b017-daa6415037a5 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:03.608842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864681a-884d-4ce8-b017-daa6415037a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:03.609556 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864681a-884d-4ce8-b017-daa6415037a5 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:03.610394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864681a-884d-4ce8-b017-daa6415037a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:03.610774 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864681a-884d-4ce8-b017-daa6415037a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:03.612254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864681a-884d-4ce8-b017-daa6415037a5 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:03.615512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e79b94-f4a6-4723-84b2-ef1f22b4c616 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:03.621364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 416/812] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:37:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:37:03.627008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864681a-884d-4ce8-b017-daa6415037a5 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:03.662529 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ef2f8-9dd5-4306-a01f-d7b57ad2f449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:03.663277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ef2f8-9dd5-4306-a01f-d7b57ad2f449 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:03.663621 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ef2f8-9dd5-4306-a01f-d7b57ad2f449 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:03.663986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ef2f8-9dd5-4306-a01f-d7b57ad2f449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:03.667321 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:03.676482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ef2f8-9dd5-4306-a01f-d7b57ad2f449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:03.678057 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ef2f8-9dd5-4306-a01f-d7b57ad2f449 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-900ef2f8-9dd5-4306-a01f-d7b57ad2f449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:37:03.693602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:03.694836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:37:03.695935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:37:03.695935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:37:03.695935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:03.699909 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 397/813] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:37:03.702627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ef2f8-9dd5-4306-a01f-d7b57ad2f449 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:03.754267 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864681a-884d-4ce8-b017-daa6415037a5 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:03.794926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95773d-212e-4441-969e-87224bb3fdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:03.795955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95773d-212e-4441-969e-87224bb3fdfc None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:03.796072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95773d-212e-4441-969e-87224bb3fdfc None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:03.796417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95773d-212e-4441-969e-87224bb3fdfc None admin] PATH_INFO: `/v3/users/e1a2d86b225045659dc8d397f1ea5180` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:03.810629 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:03.811660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af95773d-212e-4441-969e-87224bb3fdfc None admin] RBAC: Authorizing `identity:get_user(user_id=e1a2d86b225045659dc8d397f1ea5180)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:03.813209 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af95773d-212e-4441-969e-87224bb3fdfc None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:03.819928 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 417/814] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:37:03 2022] GET /identity/v3/users/e1a2d86b225045659dc8d397f1ea5180 => generated 285 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:37:03.821708 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95773d-212e-4441-969e-87224bb3fdfc None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:03.823037 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ef2f8-9dd5-4306-a01f-d7b57ad2f449 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:03.878624 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e6c86-19fc-4370-9b16-9437e8d1a081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:03.879255 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e6c86-19fc-4370-9b16-9437e8d1a081 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:03.879531 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e6c86-19fc-4370-9b16-9437e8d1a081 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:03.879798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e6c86-19fc-4370-9b16-9437e8d1a081 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:03.880523 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e6c86-19fc-4370-9b16-9437e8d1a081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:03.880825 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e6c86-19fc-4370-9b16-9437e8d1a081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:03.882257 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e6c86-19fc-4370-9b16-9437e8d1a081 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:03.896048 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 398/815] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:37:03.948200 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:03.960880 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95773d-212e-4441-969e-87224bb3fdfc None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:04.003088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705ebb4-fed7-43bd-8ff2-ce51e86817e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:04.003762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705ebb4-fed7-43bd-8ff2-ce51e86817e2 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:04.004068 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705ebb4-fed7-43bd-8ff2-ce51e86817e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:04.004333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705ebb4-fed7-43bd-8ff2-ce51e86817e2 None admin] PATH_INFO: `/v3/projects/795933040ac4472588d5c7ddb0dcac20` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:04.005652 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705ebb4-fed7-43bd-8ff2-ce51e86817e2 None admin] RBAC: Authorizing `identity:get_project(project_id=795933040ac4472588d5c7ddb0dcac20)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:04.007115 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705ebb4-fed7-43bd-8ff2-ce51e86817e2 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:04.009065 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 418/816] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:37:03 2022] GET /identity/v3/projects/795933040ac4472588d5c7ddb0dcac20 => generated 311 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:37:04.015285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e6c86-19fc-4370-9b16-9437e8d1a081 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:04.128806 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e6c86-19fc-4370-9b16-9437e8d1a081 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:04.163838 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996a6e2-b803-41f8-a752-816488f9216c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:04.164624 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996a6e2-b803-41f8-a752-816488f9216c None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:04.164876 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996a6e2-b803-41f8-a752-816488f9216c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:04.165117 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996a6e2-b803-41f8-a752-816488f9216c None admin] PATH_INFO: `/v3/projects/795933040ac4472588d5c7ddb0dcac20/users/e1a2d86b225045659dc8d397f1ea5180/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:04.167151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996a6e2-b803-41f8-a752-816488f9216c None admin] RBAC: Authorizing `identity:create_grant(project_id=795933040ac4472588d5c7ddb0dcac20, user_id=e1a2d86b225045659dc8d397f1ea5180, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:04.168812 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996a6e2-b803-41f8-a752-816488f9216c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:04.186093 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 399/817] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:37:04 2022] PUT /identity/v3/projects/795933040ac4472588d5c7ddb0dcac20/users/e1a2d86b225045659dc8d397f1ea5180/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:37:05.127910 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d8208-970b-4631-b254-d2a1d38cc658 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:05.131692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d8208-970b-4631-b254-d2a1d38cc658 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:05.131692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d8208-970b-4631-b254-d2a1d38cc658 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:05.131692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 419/818] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:05.135615 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c904f-88d8-41a9-84d9-cbf680685d47 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:05.136103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c904f-88d8-41a9-84d9-cbf680685d47 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:05.136199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c904f-88d8-41a9-84d9-cbf680685d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:05.155151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ad8c904f-88d8-41a9-84d9-cbf680685d47 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:05.156306 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ad8c904f-88d8-41a9-84d9-cbf680685d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:05.164211 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:05.226411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 400/819] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:05.493535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d8208-970b-4631-b254-d2a1d38cc658 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:05.596427 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d8208-970b-4631-b254-d2a1d38cc658 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:05.631715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6d815-e011-437f-9360-d5b4084d02a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:05.632363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6d815-e011-437f-9360-d5b4084d02a4 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:05.632638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6d815-e011-437f-9360-d5b4084d02a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:05.632906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6d815-e011-437f-9360-d5b4084d02a4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:05.643130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6d815-e011-437f-9360-d5b4084d02a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-92e6d815-e011-437f-9360-d5b4084d02a4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:37:05.647847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:05.648827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:05.649961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:05.649961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:05.649961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:05.649961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:05.649961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:37:05.649961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:37:05.649961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:37:05.649961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:37:05.649961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:37:05.649961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:37:05.649961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 15 13:37:05.649961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:37:05.649961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 420/820] 10.222.0.36 () {56 vars in 1120 bytes} [Sat Oct 15 13:37:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:37:05.655243 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c904f-88d8-41a9-84d9-cbf680685d47 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:05.759193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c904f-88d8-41a9-84d9-cbf680685d47 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:05.794323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e39b71-ea62-46ee-aa90-6b8c65682d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:05.794942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e39b71-ea62-46ee-aa90-6b8c65682d3e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:05.795215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e39b71-ea62-46ee-aa90-6b8c65682d3e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:05.795482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e39b71-ea62-46ee-aa90-6b8c65682d3e None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:05.796179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e39b71-ea62-46ee-aa90-6b8c65682d3e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:05.796480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e39b71-ea62-46ee-aa90-6b8c65682d3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:05.797933 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e39b71-ea62-46ee-aa90-6b8c65682d3e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:05.806527 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 401/821] 10.222.0.36 () {56 vars in 1115 bytes} [Sat Oct 15 13:37:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:05.812946 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6d815-e011-437f-9360-d5b4084d02a4 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:05.835961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:05.917566 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6d815-e011-437f-9360-d5b4084d02a4 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:05.953586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c9f71-0e19-4f52-93ae-8a6e849d31f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:05.954375 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c9f71-0e19-4f52-93ae-8a6e849d31f6 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:05.954722 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c9f71-0e19-4f52-93ae-8a6e849d31f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:05.954995 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c9f71-0e19-4f52-93ae-8a6e849d31f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:05.967497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c9f71-0e19-4f52-93ae-8a6e849d31f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:05.969064 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c9f71-0e19-4f52-93ae-8a6e849d31f6 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-9f2c9f71-0e19-4f52-93ae-8a6e849d31f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:37:05.986110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:05.987304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:37:05.988233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:37:05.988233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:37:05.988233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:37:05.988233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 421/822] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:37:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:37:05.993550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e39b71-ea62-46ee-aa90-6b8c65682d3e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:05.997605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:06.094566 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e39b71-ea62-46ee-aa90-6b8c65682d3e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:06.128776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf9457-8f30-453e-8244-72ff59df42a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:06.129386 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf9457-8f30-453e-8244-72ff59df42a9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:06.129651 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf9457-8f30-453e-8244-72ff59df42a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:06.129909 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf9457-8f30-453e-8244-72ff59df42a9 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:06.130539 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf9457-8f30-453e-8244-72ff59df42a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:06.130832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf9457-8f30-453e-8244-72ff59df42a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:06.132236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf9457-8f30-453e-8244-72ff59df42a9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:06.145002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 402/823] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:06.151056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c9f71-0e19-4f52-93ae-8a6e849d31f6 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:06.227492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:06.261146 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c9f71-0e19-4f52-93ae-8a6e849d31f6 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:06.299335 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e2716-4434-4365-a60b-a1d8aabbc042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:06.300022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e2716-4434-4365-a60b-a1d8aabbc042 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:06.300304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e2716-4434-4365-a60b-a1d8aabbc042 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:06.300574 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e2716-4434-4365-a60b-a1d8aabbc042 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:06.309351 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e2716-4434-4365-a60b-a1d8aabbc042 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:06.310790 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e2716-4434-4365-a60b-a1d8aabbc042 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-2a8e2716-4434-4365-a60b-a1d8aabbc042 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:37:06.322301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:06.323492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:06.324714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:06.324714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:06.324714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:37:06.324714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:37:06.324714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:37:06.324714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:37:06.324714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:37:06.324714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:37:06.324714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 15 13:37:06.324714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:37:06.324714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 422/824] 10.222.0.36 () {56 vars in 1093 bytes} [Sat Oct 15 13:37:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:37:06.329371 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90ec77-b260-488c-8bef-e8ec3312a4c3 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:06.329675 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90ec77-b260-488c-8bef-e8ec3312a4c3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:06.329787 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e2716-4434-4365-a60b-a1d8aabbc042 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:06.330129 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90ec77-b260-488c-8bef-e8ec3312a4c3 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:06.335718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 403/825] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:06.337013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770671c-5f5e-4b12-b147-b471593aac2b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:06.337290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770671c-5f5e-4b12-b147-b471593aac2b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:06.337563 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770671c-5f5e-4b12-b147-b471593aac2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:06.355668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f770671c-5f5e-4b12-b147-b471593aac2b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:06.357008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f770671c-5f5e-4b12-b147-b471593aac2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:06.419931 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:06.430148 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 404/826] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:06.447172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e2716-4434-4365-a60b-a1d8aabbc042 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:06.476658 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:06.482297 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d9eda-dae8-4086-98e5-ac359be6035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:06.483130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d9eda-dae8-4086-98e5-ac359be6035f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:06.483564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d9eda-dae8-4086-98e5-ac359be6035f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:06.483838 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d9eda-dae8-4086-98e5-ac359be6035f None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:06.484944 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d9eda-dae8-4086-98e5-ac359be6035f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:06.485586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d9eda-dae8-4086-98e5-ac359be6035f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:06.487187 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d9eda-dae8-4086-98e5-ac359be6035f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:06.506049 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 423/827] 10.222.0.36 () {56 vars in 1133 bytes} [Sat Oct 15 13:37:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:37:06.511745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770671c-5f5e-4b12-b147-b471593aac2b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:06.612354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770671c-5f5e-4b12-b147-b471593aac2b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:06.648870 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ca444-4201-4803-bacb-53312bb0387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:06.649504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ca444-4201-4803-bacb-53312bb0387e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:06.649780 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ca444-4201-4803-bacb-53312bb0387e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:06.650048 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ca444-4201-4803-bacb-53312bb0387e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:06.703112 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d9eda-dae8-4086-98e5-ac359be6035f None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:06.719762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:06.727672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ca444-4201-4803-bacb-53312bb0387e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:06.729186 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ca444-4201-4803-bacb-53312bb0387e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:06.814538 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d9eda-dae8-4086-98e5-ac359be6035f None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:06.836813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 405/828] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:37:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 13:37:06.842818 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ca444-4201-4803-bacb-53312bb0387e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:06.851334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2140775-765b-4d9a-abcf-956fd42b8072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:06.852069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2140775-765b-4d9a-abcf-956fd42b8072 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:06.852297 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2140775-765b-4d9a-abcf-956fd42b8072 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:06.852569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2140775-765b-4d9a-abcf-956fd42b8072 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:06.863556 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2140775-765b-4d9a-abcf-956fd42b8072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-f2140775-765b-4d9a-abcf-956fd42b8072 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:37:06.868302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:06.869483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:06.870904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:06.870904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:06.870904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:06.870904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:06.870904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:37:06.870904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:37:06.870904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:37:06.870904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:37:06.870904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:37:06.870904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:37:06.870904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:37:06.870904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:37:06.870904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 424/829] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:37:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:37:06.877130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2140775-765b-4d9a-abcf-956fd42b8072 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:06.928501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:06.945499 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:06.953083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ca444-4201-4803-bacb-53312bb0387e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:06.982312 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2140775-765b-4d9a-abcf-956fd42b8072 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:06.991203 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12f128-cb20-4ea0-b0c7-f6ed1d723325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:06.991854 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12f128-cb20-4ea0-b0c7-f6ed1d723325 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:06.992180 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12f128-cb20-4ea0-b0c7-f6ed1d723325 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:06.992447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12f128-cb20-4ea0-b0c7-f6ed1d723325 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:07.003638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12f128-cb20-4ea0-b0c7-f6ed1d723325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:07.005178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12f128-cb20-4ea0-b0c7-f6ed1d723325 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:07.020403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fe959-681b-48ab-a44b-359b834852d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-6b12f128-cb20-4ea0-b0c7-f6ed1d723325 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:37:07.020636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:07.021605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:37:07.022495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:37:07.022495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:37:07.022495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:07.022664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fe959-681b-48ab-a44b-359b834852d3 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:07.022664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fe959-681b-48ab-a44b-359b834852d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:07.022664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fe959-681b-48ab-a44b-359b834852d3 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:07.022883 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 406/830] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:37:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:37:07.022956 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fe959-681b-48ab-a44b-359b834852d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:07.022956 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fe959-681b-48ab-a44b-359b834852d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:07.023860 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fe959-681b-48ab-a44b-359b834852d3 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:07.028010 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12f128-cb20-4ea0-b0c7-f6ed1d723325 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:07.034554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 425/831] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:37:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:37:07.040664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fe959-681b-48ab-a44b-359b834852d3 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:07.134961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12f128-cb20-4ea0-b0c7-f6ed1d723325 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:07.145176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fe959-681b-48ab-a44b-359b834852d3 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:07.158965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:07.171510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086315c-efa5-41b8-809a-d0019b58dfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:07.172168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086315c-efa5-41b8-809a-d0019b58dfa1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:07.172448 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086315c-efa5-41b8-809a-d0019b58dfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:07.172754 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086315c-efa5-41b8-809a-d0019b58dfa1 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:07.173470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086315c-efa5-41b8-809a-d0019b58dfa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:07.173770 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086315c-efa5-41b8-809a-d0019b58dfa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:07.175155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086315c-efa5-41b8-809a-d0019b58dfa1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:07.181258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf8cc3-8add-4339-bd0c-8b32087d4f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:07.181898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf8cc3-8add-4339-bd0c-8b32087d4f59 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:07.182176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf8cc3-8add-4339-bd0c-8b32087d4f59 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:07.182467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf8cc3-8add-4339-bd0c-8b32087d4f59 None admin] PATH_INFO: `/v3/users/e1a2d86b225045659dc8d397f1ea5180` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:07.183602 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf8cc3-8add-4339-bd0c-8b32087d4f59 None admin] RBAC: Authorizing `identity:get_user(user_id=e1a2d86b225045659dc8d397f1ea5180)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:07.185065 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf8cc3-8add-4339-bd0c-8b32087d4f59 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:07.186671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 426/832] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:37:07 2022] GET /identity/v3/users/e1a2d86b225045659dc8d397f1ea5180 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:37:07.187400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 407/833] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:07.192906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf8cc3-8add-4339-bd0c-8b32087d4f59 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:07.193052 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086315c-efa5-41b8-809a-d0019b58dfa1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:07.201776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:07.294824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086315c-efa5-41b8-809a-d0019b58dfa1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:07.295107 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf8cc3-8add-4339-bd0c-8b32087d4f59 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:07.335431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12292cb3-537f-4545-9c67-79191cc826d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:07.335674 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735541c-d66a-4c9a-813d-7f5a55c155a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:07.336120 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12292cb3-537f-4545-9c67-79191cc826d0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:07.336245 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735541c-d66a-4c9a-813d-7f5a55c155a8 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:07.336393 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735541c-d66a-4c9a-813d-7f5a55c155a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:07.336490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12292cb3-537f-4545-9c67-79191cc826d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:07.336653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12292cb3-537f-4545-9c67-79191cc826d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:07.336756 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735541c-d66a-4c9a-813d-7f5a55c155a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:07.344640 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12292cb3-537f-4545-9c67-79191cc826d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:07.346520 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12292cb3-537f-4545-9c67-79191cc826d0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-12292cb3-537f-4545-9c67-79191cc826d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:37:07.359929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:07.361218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:07.362122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:37:07.362122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:37:07.362122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:37:07.362122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:37:07.362122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:37:07.362122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:37:07.362122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:37:07.362122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:07.362122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 408/834] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:37:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:37:07.367972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12292cb3-537f-4545-9c67-79191cc826d0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:07.393093 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:07.417551 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735541c-d66a-4c9a-813d-7f5a55c155a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:07.419021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735541c-d66a-4c9a-813d-7f5a55c155a8 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:07.442248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:07.480442 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12292cb3-537f-4545-9c67-79191cc826d0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:07.522130 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cddf7-a7e0-474f-b397-a7bee9c1da02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:07.522772 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cddf7-a7e0-474f-b397-a7bee9c1da02 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:07.523040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cddf7-a7e0-474f-b397-a7bee9c1da02 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:07.523308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cddf7-a7e0-474f-b397-a7bee9c1da02 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:07.524228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cddf7-a7e0-474f-b397-a7bee9c1da02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:07.524584 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cddf7-a7e0-474f-b397-a7bee9c1da02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:07.526026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8cddf7-a7e0-474f-b397-a7bee9c1da02 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:07.529384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 427/835] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:37:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 15 13:37:07.535605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735541c-d66a-4c9a-813d-7f5a55c155a8 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:07.544448 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 409/836] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:37:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:07.545985 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cddf7-a7e0-474f-b397-a7bee9c1da02 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:07.615278 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:07.634389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735541c-d66a-4c9a-813d-7f5a55c155a8 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:07.648721 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cddf7-a7e0-474f-b397-a7bee9c1da02 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:07.668155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ec39b-8e0e-4cac-b365-f035cc40a1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:07.668807 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ec39b-8e0e-4cac-b365-f035cc40a1be None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:07.669071 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ec39b-8e0e-4cac-b365-f035cc40a1be None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:07.669343 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ec39b-8e0e-4cac-b365-f035cc40a1be None admin] PATH_INFO: `/v3/projects/795933040ac4472588d5c7ddb0dcac20` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:07.670572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ec39b-8e0e-4cac-b365-f035cc40a1be None admin] RBAC: Authorizing `identity:get_project(project_id=795933040ac4472588d5c7ddb0dcac20)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:07.672074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ec39b-8e0e-4cac-b365-f035cc40a1be None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:07.673836 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 428/837] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:37:07 2022] GET /identity/v3/projects/795933040ac4472588d5c7ddb0dcac20 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:37:07.674453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:07.680479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ec39b-8e0e-4cac-b365-f035cc40a1be None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:07.685195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e09f76-34a5-4f26-aaf0-cf2060b2afdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:07.685811 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e09f76-34a5-4f26-aaf0-cf2060b2afdc None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:07.686087 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e09f76-34a5-4f26-aaf0-cf2060b2afdc None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:07.686358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e09f76-34a5-4f26-aaf0-cf2060b2afdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:07.687085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e09f76-34a5-4f26-aaf0-cf2060b2afdc None admin] RBAC: Adding query filter params (role.id=5bd4f51ce7594d178e68f303bb34cced, scope.project.id=2a5a64cc8aa04972bb464407cb0312b4, user.id=2e13692c1e83413a909d99bb7db2e4b8) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:07.687389 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e09f76-34a5-4f26-aaf0-cf2060b2afdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:07.689039 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e09f76-34a5-4f26-aaf0-cf2060b2afdc None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:07.695179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 410/838] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:37:07 2022] GET /identity/v3/role_assignments?user.id=2e13692c1e83413a909d99bb7db2e4b8&scope.project.id=2a5a64cc8aa04972bb464407cb0312b4&role.id=5bd4f51ce7594d178e68f303bb34cced => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:07.788967 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ec39b-8e0e-4cac-b365-f035cc40a1be None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:07.828295 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77503d90-1f0a-4944-ab98-1cf04850cb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:07.828933 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77503d90-1f0a-4944-ab98-1cf04850cb96 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:07.829286 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77503d90-1f0a-4944-ab98-1cf04850cb96 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:07.829645 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77503d90-1f0a-4944-ab98-1cf04850cb96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:07.830418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77503d90-1f0a-4944-ab98-1cf04850cb96 None admin] RBAC: Adding query filter params (role.id=57740dc2ebe1429294f20dd51592b82a, scope.project.id=795933040ac4472588d5c7ddb0dcac20, user.id=e1a2d86b225045659dc8d397f1ea5180) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:07.830729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77503d90-1f0a-4944-ab98-1cf04850cb96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:07.832261 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77503d90-1f0a-4944-ab98-1cf04850cb96 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:07.838995 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 429/839] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:37:07 2022] GET /identity/v3/role_assignments?user.id=e1a2d86b225045659dc8d397f1ea5180&scope.project.id=795933040ac4472588d5c7ddb0dcac20&role.id=57740dc2ebe1429294f20dd51592b82a => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:37:09.990903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f11d69-b0f4-4560-bc99-aacf82c365f3 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:09.991525 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f11d69-b0f4-4560-bc99-aacf82c365f3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:09.991525 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f11d69-b0f4-4560-bc99-aacf82c365f3 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:09.992800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 411/840] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:09.998543 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7cf9b-d7af-400c-b10c-aba812d0cac5 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:09.999050 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7cf9b-d7af-400c-b10c-aba812d0cac5 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:09.999156 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7cf9b-d7af-400c-b10c-aba812d0cac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:10.020780 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-b9d7cf9b-d7af-400c-b10c-aba812d0cac5 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:10.021969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b9d7cf9b-d7af-400c-b10c-aba812d0cac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:10.030576 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:10.094230 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 430/841] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:10.358548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f11d69-b0f4-4560-bc99-aacf82c365f3 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:10.480447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f11d69-b0f4-4560-bc99-aacf82c365f3 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:10.520386 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35668663-d58a-40cb-9239-30bc939685d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:10.521021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35668663-d58a-40cb-9239-30bc939685d4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:10.522065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35668663-d58a-40cb-9239-30bc939685d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:10.522065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35668663-d58a-40cb-9239-30bc939685d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:10.522761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35668663-d58a-40cb-9239-30bc939685d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:10.524241 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35668663-d58a-40cb-9239-30bc939685d4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:10.525847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 412/842] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:37:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:10.532754 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7cf9b-d7af-400c-b10c-aba812d0cac5 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:10.644722 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7cf9b-d7af-400c-b10c-aba812d0cac5 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:10.683051 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a307e73-5f2e-4f74-8b0b-3d014ac38c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:10.683666 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a307e73-5f2e-4f74-8b0b-3d014ac38c40 None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:10.683966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a307e73-5f2e-4f74-8b0b-3d014ac38c40 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:10.684235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a307e73-5f2e-4f74-8b0b-3d014ac38c40 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:10.684889 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a307e73-5f2e-4f74-8b0b-3d014ac38c40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:10.686437 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a307e73-5f2e-4f74-8b0b-3d014ac38c40 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:10.703805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:10.711346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 431/843] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:37:10 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:37:12.969278 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917f86a-fa34-4e08-9584-f5e4b6221517 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:12.969915 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917f86a-fa34-4e08-9584-f5e4b6221517 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:12.969915 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917f86a-fa34-4e08-9584-f5e4b6221517 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:12.971732 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 413/844] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:12.977789 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1909ef-d24b-46a5-b3d0-97b53a431881 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:12.978276 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1909ef-d24b-46a5-b3d0-97b53a431881 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:12.978376 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1909ef-d24b-46a5-b3d0-97b53a431881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:12.998090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-ac1909ef-d24b-46a5-b3d0-97b53a431881 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:12.999266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ac1909ef-d24b-46a5-b3d0-97b53a431881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:13.067734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 432/845] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:13.330181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917f86a-fa34-4e08-9584-f5e4b6221517 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:13.394015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:13.437759 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917f86a-fa34-4e08-9584-f5e4b6221517 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:13.474119 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4d77e-90bf-43af-8616-e064ef4b63a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:13.474748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4d77e-90bf-43af-8616-e064ef4b63a4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:13.475018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4d77e-90bf-43af-8616-e064ef4b63a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:13.475281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4d77e-90bf-43af-8616-e064ef4b63a4 None admin] PATH_INFO: `/v3/roles/8a864244c21042b88accaa76b9117665` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:13.476815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4d77e-90bf-43af-8616-e064ef4b63a4 None admin] RBAC: Authorizing `identity:get_role(role_id=8a864244c21042b88accaa76b9117665)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:13.478200 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4d77e-90bf-43af-8616-e064ef4b63a4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:13.479546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 414/846] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:37:13 2022] GET /identity/v3/roles/8a864244c21042b88accaa76b9117665 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:13.486404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1909ef-d24b-46a5-b3d0-97b53a431881 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:13.589911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1909ef-d24b-46a5-b3d0-97b53a431881 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:13.617366 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:13.628004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6a6ba-df63-45e4-a1e7-47e7dea57d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:13.628639 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6a6ba-df63-45e4-a1e7-47e7dea57d14 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:13.628913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6a6ba-df63-45e4-a1e7-47e7dea57d14 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:13.629175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6a6ba-df63-45e4-a1e7-47e7dea57d14 None admin] PATH_INFO: `/v3/users/5f9ad81234c04f99a95755f4649a3110` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:13.643589 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6a6ba-df63-45e4-a1e7-47e7dea57d14 None admin] RBAC: Authorizing `identity:get_user(user_id=5f9ad81234c04f99a95755f4649a3110)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:13.645087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6a6ba-df63-45e4-a1e7-47e7dea57d14 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:13.646840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 433/847] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:37:13 2022] GET /identity/v3/users/5f9ad81234c04f99a95755f4649a3110 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:37:13.654303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4d77e-90bf-43af-8616-e064ef4b63a4 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:13.761509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4d77e-90bf-43af-8616-e064ef4b63a4 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:13.790956 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:13.800774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d308108-14bb-4c58-bbde-387d5971b3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:13.801441 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d308108-14bb-4c58-bbde-387d5971b3f4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:13.801718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d308108-14bb-4c58-bbde-387d5971b3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:13.801992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d308108-14bb-4c58-bbde-387d5971b3f4 None admin] PATH_INFO: `/v3/projects/795933040ac4472588d5c7ddb0dcac20` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:13.803219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d308108-14bb-4c58-bbde-387d5971b3f4 None admin] RBAC: Authorizing `identity:get_project(project_id=795933040ac4472588d5c7ddb0dcac20)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:13.804749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d308108-14bb-4c58-bbde-387d5971b3f4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:13.806511 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 415/848] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:37:13 2022] GET /identity/v3/projects/795933040ac4472588d5c7ddb0dcac20 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:13.813037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6a6ba-df63-45e4-a1e7-47e7dea57d14 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:13.927591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6a6ba-df63-45e4-a1e7-47e7dea57d14 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:13.964790 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5529ff-e988-4ee5-abba-9f5b57f189e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:13.965458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5529ff-e988-4ee5-abba-9f5b57f189e9 None admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:13.965744 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5529ff-e988-4ee5-abba-9f5b57f189e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:13.966015 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5529ff-e988-4ee5-abba-9f5b57f189e9 None admin] PATH_INFO: `/v3/projects/795933040ac4472588d5c7ddb0dcac20/users/5f9ad81234c04f99a95755f4649a3110/roles/8a864244c21042b88accaa76b9117665` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:13.968381 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5529ff-e988-4ee5-abba-9f5b57f189e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=795933040ac4472588d5c7ddb0dcac20, user_id=5f9ad81234c04f99a95755f4649a3110, role_id=8a864244c21042b88accaa76b9117665)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:13.969911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5529ff-e988-4ee5-abba-9f5b57f189e9 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:13.987059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 434/849] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:37:13 2022] PUT /identity/v3/projects/795933040ac4472588d5c7ddb0dcac20/users/5f9ad81234c04f99a95755f4649a3110/roles/8a864244c21042b88accaa76b9117665 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:37:16.124645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ecb9d-a06f-4bfe-bb0b-3e47feaddcdb None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:16.125224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ecb9d-a06f-4bfe-bb0b-3e47feaddcdb None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:16.125224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ecb9d-a06f-4bfe-bb0b-3e47feaddcdb None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:16.126567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 416/850] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:16.132029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792407e7-3a1c-4e32-99f9-77f8a5b5de2b None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:16.132482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792407e7-3a1c-4e32-99f9-77f8a5b5de2b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:16.132577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792407e7-3a1c-4e32-99f9-77f8a5b5de2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:16.151913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-792407e7-3a1c-4e32-99f9-77f8a5b5de2b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:16.153045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-792407e7-3a1c-4e32-99f9-77f8a5b5de2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:16.197446 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:16.222784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 435/851] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:16.493148 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ecb9d-a06f-4bfe-bb0b-3e47feaddcdb None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:16.605908 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ecb9d-a06f-4bfe-bb0b-3e47feaddcdb None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:16.642504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f02506-6635-4967-836f-f7fea39e261b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:16.643138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f02506-6635-4967-836f-f7fea39e261b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:16.643413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f02506-6635-4967-836f-f7fea39e261b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:16.643681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f02506-6635-4967-836f-f7fea39e261b None admin] PATH_INFO: `/v3/roles/8a864244c21042b88accaa76b9117665` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:16.645146 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f02506-6635-4967-836f-f7fea39e261b None admin] RBAC: Authorizing `identity:get_role(role_id=8a864244c21042b88accaa76b9117665)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:16.646528 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f02506-6635-4967-836f-f7fea39e261b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:16.648040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 417/852] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:37:16 2022] GET /identity/v3/roles/8a864244c21042b88accaa76b9117665 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:16.654170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792407e7-3a1c-4e32-99f9-77f8a5b5de2b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:16.768684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792407e7-3a1c-4e32-99f9-77f8a5b5de2b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:16.805535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65375e92-165a-4586-b82b-8bd8cb05ad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:16.806196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65375e92-165a-4586-b82b-8bd8cb05ad88 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:16.806456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65375e92-165a-4586-b82b-8bd8cb05ad88 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:16.806730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65375e92-165a-4586-b82b-8bd8cb05ad88 None admin] PATH_INFO: `/v3/users/5f9ad81234c04f99a95755f4649a3110` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:16.807980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65375e92-165a-4586-b82b-8bd8cb05ad88 None admin] RBAC: Authorizing `identity:get_user(user_id=5f9ad81234c04f99a95755f4649a3110)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:16.809405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65375e92-165a-4586-b82b-8bd8cb05ad88 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:16.811009 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 436/853] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:37:16 2022] GET /identity/v3/users/5f9ad81234c04f99a95755f4649a3110 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:16.817534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f02506-6635-4967-836f-f7fea39e261b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:16.889898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:16.934486 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f02506-6635-4967-836f-f7fea39e261b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:16.968966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e06af6-620a-4d9a-925a-880211e800d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:16.969572 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e06af6-620a-4d9a-925a-880211e800d8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:16.969911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e06af6-620a-4d9a-925a-880211e800d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:16.970182 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e06af6-620a-4d9a-925a-880211e800d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:17.040925 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e06af6-620a-4d9a-925a-880211e800d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:17.042296 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e06af6-620a-4d9a-925a-880211e800d8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:17.103265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:17.144065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 418/854] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:37:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 13:37:17.150306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65375e92-165a-4586-b82b-8bd8cb05ad88 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:17.197182 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:17.253684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65375e92-165a-4586-b82b-8bd8cb05ad88 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:17.288315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e6a3e-f75b-41aa-a6d1-71e853627884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:17.288947 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e6a3e-f75b-41aa-a6d1-71e853627884 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:17.289221 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e6a3e-f75b-41aa-a6d1-71e853627884 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:17.289488 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e6a3e-f75b-41aa-a6d1-71e853627884 None admin] PATH_INFO: `/v3/projects/795933040ac4472588d5c7ddb0dcac20` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:17.290653 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e6a3e-f75b-41aa-a6d1-71e853627884 None admin] RBAC: Authorizing `identity:get_project(project_id=795933040ac4472588d5c7ddb0dcac20)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:17.292177 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e6a3e-f75b-41aa-a6d1-71e853627884 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:17.293753 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 437/855] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:37:17 2022] GET /identity/v3/projects/795933040ac4472588d5c7ddb0dcac20 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:17.300640 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e06af6-620a-4d9a-925a-880211e800d8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:17.403774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e06af6-620a-4d9a-925a-880211e800d8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:17.440134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373706c-91e5-43a7-b6ea-11a32c3615cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:17.440739 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373706c-91e5-43a7-b6ea-11a32c3615cc None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:17.441012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373706c-91e5-43a7-b6ea-11a32c3615cc None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:17.441289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373706c-91e5-43a7-b6ea-11a32c3615cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:17.441989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373706c-91e5-43a7-b6ea-11a32c3615cc None admin] RBAC: Adding query filter params (role.id=8a864244c21042b88accaa76b9117665, scope.project.id=795933040ac4472588d5c7ddb0dcac20, user.id=5f9ad81234c04f99a95755f4649a3110) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:17.442293 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373706c-91e5-43a7-b6ea-11a32c3615cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:17.443736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373706c-91e5-43a7-b6ea-11a32c3615cc None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:17.449903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 419/856] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:37:17 2022] GET /identity/v3/role_assignments?user.id=5f9ad81234c04f99a95755f4649a3110&scope.project.id=795933040ac4472588d5c7ddb0dcac20&role.id=8a864244c21042b88accaa76b9117665 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:19.596596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7430e3-1880-4755-89fe-0de4d5cfa6b0 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:19.597193 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7430e3-1880-4755-89fe-0de4d5cfa6b0 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:19.597193 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7430e3-1880-4755-89fe-0de4d5cfa6b0 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:19.598562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 438/857] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:19.604895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008d4fd-cfb6-419a-b16a-5be4682a9690 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:19.605519 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008d4fd-cfb6-419a-b16a-5be4682a9690 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:19.605519 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008d4fd-cfb6-419a-b16a-5be4682a9690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:19.625807 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f008d4fd-cfb6-419a-b16a-5be4682a9690 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:19.627122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f008d4fd-cfb6-419a-b16a-5be4682a9690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:19.651192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:19.702570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 420/858] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:19.966793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7430e3-1880-4755-89fe-0de4d5cfa6b0 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:20.072668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7430e3-1880-4755-89fe-0de4d5cfa6b0 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:20.086353 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:20.109559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45582d-6cc5-499f-bd43-7a24bbacb67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:20.110184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45582d-6cc5-499f-bd43-7a24bbacb67a None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:20.110581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45582d-6cc5-499f-bd43-7a24bbacb67a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:20.110792 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45582d-6cc5-499f-bd43-7a24bbacb67a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:20.111967 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45582d-6cc5-499f-bd43-7a24bbacb67a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:20.113377 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45582d-6cc5-499f-bd43-7a24bbacb67a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:20.114966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 439/859] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:37:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:20.121145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008d4fd-cfb6-419a-b16a-5be4682a9690 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:20.224457 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008d4fd-cfb6-419a-b16a-5be4682a9690 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:20.260495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c66f76-073a-41dc-8393-f1560d00430a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:20.261109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66f76-073a-41dc-8393-f1560d00430a None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:20.261382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66f76-073a-41dc-8393-f1560d00430a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:20.261651 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66f76-073a-41dc-8393-f1560d00430a None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:20.262296 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c66f76-073a-41dc-8393-f1560d00430a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:20.263740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c66f76-073a-41dc-8393-f1560d00430a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:20.293364 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 421/860] 10.222.0.36 () {60 vars in 1141 bytes} [Sat Oct 15 13:37:20 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:37:22.471601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17e01a-60c0-466a-8e17-1383e3eeca19 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:22.472183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17e01a-60c0-466a-8e17-1383e3eeca19 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:22.472183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17e01a-60c0-466a-8e17-1383e3eeca19 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:22.473520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 440/861] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:22.479753 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bb642-be56-4736-9ec9-f37669324bbd None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:22.480192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bb642-be56-4736-9ec9-f37669324bbd None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:22.480341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bb642-be56-4736-9ec9-f37669324bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:22.505018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5f9bb642-be56-4736-9ec9-f37669324bbd None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:22.506213 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5f9bb642-be56-4736-9ec9-f37669324bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:22.508260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:22.584765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 422/862] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:22.849943 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17e01a-60c0-466a-8e17-1383e3eeca19 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:22.965423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17e01a-60c0-466a-8e17-1383e3eeca19 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:23.009745 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff312143-e739-4fc7-b15e-c7b3bd9e59aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:23.010405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff312143-e739-4fc7-b15e-c7b3bd9e59aa None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:23.010685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff312143-e739-4fc7-b15e-c7b3bd9e59aa None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:23.010955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff312143-e739-4fc7-b15e-c7b3bd9e59aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:23.012310 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff312143-e739-4fc7-b15e-c7b3bd9e59aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:23.013772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff312143-e739-4fc7-b15e-c7b3bd9e59aa None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:23.015448 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 441/863] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:37:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:23.021849 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bb642-be56-4736-9ec9-f37669324bbd None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:23.128076 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bb642-be56-4736-9ec9-f37669324bbd None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:23.162964 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bda078-bc3d-4742-97e9-6baffeea837c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:23.163576 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bda078-bc3d-4742-97e9-6baffeea837c None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:23.163851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bda078-bc3d-4742-97e9-6baffeea837c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:23.164148 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bda078-bc3d-4742-97e9-6baffeea837c None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:23.164791 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bda078-bc3d-4742-97e9-6baffeea837c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:23.166252 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bda078-bc3d-4742-97e9-6baffeea837c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:23.176575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:23.190291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 423/864] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:37:23 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:37:25.329835 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be30029-29e1-4874-bbff-64edad0cc635 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:25.330387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be30029-29e1-4874-bbff-64edad0cc635 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:25.330387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be30029-29e1-4874-bbff-64edad0cc635 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:25.331618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 442/865] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:25.336903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a30b-94df-4d5e-b31c-cba44b19ae66 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:25.337347 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a30b-94df-4d5e-b31c-cba44b19ae66 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:25.337438 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a30b-94df-4d5e-b31c-cba44b19ae66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:25.355645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c2e0a30b-94df-4d5e-b31c-cba44b19ae66 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:25.356835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c2e0a30b-94df-4d5e-b31c-cba44b19ae66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:25.424086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 424/866] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:25.685547 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be30029-29e1-4874-bbff-64edad0cc635 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:25.740278 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:25.794711 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be30029-29e1-4874-bbff-64edad0cc635 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:25.830789 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca11bc-c231-4c5b-b28a-45c5204d32f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:25.831401 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca11bc-c231-4c5b-b28a-45c5204d32f8 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:25.831672 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca11bc-c231-4c5b-b28a-45c5204d32f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:25.831965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca11bc-c231-4c5b-b28a-45c5204d32f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:25.842680 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca11bc-c231-4c5b-b28a-45c5204d32f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-2aca11bc-c231-4c5b-b28a-45c5204d32f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:37:25.847417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:25.848640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:25.849713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:25.849713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:25.849713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:25.849713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:25.849713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:37:25.849713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:37:25.849713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:37:25.849713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:37:25.849713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:37:25.849713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:37:25.849713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:37:25.849713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:37:25.849713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 443/867] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:37:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:37:25.855380 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0a30b-94df-4d5e-b31c-cba44b19ae66 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:25.961557 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0a30b-94df-4d5e-b31c-cba44b19ae66 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:25.985222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:25.997973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e114b-3236-4599-9cec-3a7d201fadcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:25.998583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e114b-3236-4599-9cec-3a7d201fadcb None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:25.998833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e114b-3236-4599-9cec-3a7d201fadcb None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:25.999090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e114b-3236-4599-9cec-3a7d201fadcb None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:25.999744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e114b-3236-4599-9cec-3a7d201fadcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:26.000077 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e114b-3236-4599-9cec-3a7d201fadcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:26.001478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e114b-3236-4599-9cec-3a7d201fadcb None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:26.009992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 425/868] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:37:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:26.016226 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca11bc-c231-4c5b-b28a-45c5204d32f8 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:26.116714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca11bc-c231-4c5b-b28a-45c5204d32f8 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:26.126101 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:26.154530 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af399b38-0953-4c30-85fd-b65be760f59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:26.155169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af399b38-0953-4c30-85fd-b65be760f59a None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:26.155445 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af399b38-0953-4c30-85fd-b65be760f59a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:26.155714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af399b38-0953-4c30-85fd-b65be760f59a None admin] PATH_INFO: `/v3/users/82751009ca4a4cb99090d3d96520305d` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:26.169731 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af399b38-0953-4c30-85fd-b65be760f59a None admin] RBAC: Authorizing `identity:get_user(user_id=82751009ca4a4cb99090d3d96520305d)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:26.171180 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af399b38-0953-4c30-85fd-b65be760f59a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:26.172990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 444/869] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:37:26 2022] GET /identity/v3/users/82751009ca4a4cb99090d3d96520305d => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:26.179218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e114b-3236-4599-9cec-3a7d201fadcb None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:26.295467 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e114b-3236-4599-9cec-3a7d201fadcb None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:26.329419 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aac594-1a2f-4032-88fa-ce974bdec7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:26.330055 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aac594-1a2f-4032-88fa-ce974bdec7cd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:26.330335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aac594-1a2f-4032-88fa-ce974bdec7cd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:26.330600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aac594-1a2f-4032-88fa-ce974bdec7cd None admin] PATH_INFO: `/v3/projects/a532c3269a284e9587b5601358a40c93` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:26.331772 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aac594-1a2f-4032-88fa-ce974bdec7cd None admin] RBAC: Authorizing `identity:get_project(project_id=a532c3269a284e9587b5601358a40c93)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:26.333243 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aac594-1a2f-4032-88fa-ce974bdec7cd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:26.334866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 426/870] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:37:26 2022] GET /identity/v3/projects/a532c3269a284e9587b5601358a40c93 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:26.341295 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af399b38-0953-4c30-85fd-b65be760f59a None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:26.448070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af399b38-0953-4c30-85fd-b65be760f59a None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:26.490222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6380d-db4a-4bc4-8c8f-a92ab389d62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:26.490847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6380d-db4a-4bc4-8c8f-a92ab389d62c None admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:26.491101 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6380d-db4a-4bc4-8c8f-a92ab389d62c None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:26.491401 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6380d-db4a-4bc4-8c8f-a92ab389d62c None admin] PATH_INFO: `/v3/projects/a532c3269a284e9587b5601358a40c93/users/82751009ca4a4cb99090d3d96520305d/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:26.493630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6380d-db4a-4bc4-8c8f-a92ab389d62c None admin] RBAC: Authorizing `identity:create_grant(project_id=a532c3269a284e9587b5601358a40c93, user_id=82751009ca4a4cb99090d3d96520305d, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:26.495057 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6380d-db4a-4bc4-8c8f-a92ab389d62c None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:26.510495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 445/871] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:37:26 2022] PUT /identity/v3/projects/a532c3269a284e9587b5601358a40c93/users/82751009ca4a4cb99090d3d96520305d/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:37:28.659779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0974a4-d5f4-495b-961e-f251ad5a3630 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:28.660372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0974a4-d5f4-495b-961e-f251ad5a3630 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:28.660372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0974a4-d5f4-495b-961e-f251ad5a3630 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:28.661684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 427/872] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:28.667144 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17ac4f-358a-458d-89ec-050b8df3eb76 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:28.667579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17ac4f-358a-458d-89ec-050b8df3eb76 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:28.667667 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17ac4f-358a-458d-89ec-050b8df3eb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:28.687008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-1e17ac4f-358a-458d-89ec-050b8df3eb76 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:28.688217 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1e17ac4f-358a-458d-89ec-050b8df3eb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:28.709403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:28.759014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 446/873] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:29.026154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0974a4-d5f4-495b-961e-f251ad5a3630 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:29.086871 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:29.133173 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0974a4-d5f4-495b-961e-f251ad5a3630 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:29.168176 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f06e1-edff-4b02-951c-905b97625a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:29.168803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f06e1-edff-4b02-951c-905b97625a78 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:29.169123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f06e1-edff-4b02-951c-905b97625a78 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:29.169320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f06e1-edff-4b02-951c-905b97625a78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:29.181139 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f06e1-edff-4b02-951c-905b97625a78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-529f06e1-edff-4b02-951c-905b97625a78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:37:29.185969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:29.186962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:29.187803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:29.187803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:29.187803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:29.187803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:29.187803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:37:29.187803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:37:29.187803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:37:29.187803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:37:29.187803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:37:29.187803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:37:29.187803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:37:29.187803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:29.187803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 428/874] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:37:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:37:29.193590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17ac4f-358a-458d-89ec-050b8df3eb76 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:29.296636 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17ac4f-358a-458d-89ec-050b8df3eb76 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:29.330663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf38499-6380-424d-b9e5-67cc51154382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:29.331265 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf38499-6380-424d-b9e5-67cc51154382 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:29.331541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf38499-6380-424d-b9e5-67cc51154382 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:29.331805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf38499-6380-424d-b9e5-67cc51154382 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:29.332498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf38499-6380-424d-b9e5-67cc51154382 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:29.332794 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf38499-6380-424d-b9e5-67cc51154382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:29.334147 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf38499-6380-424d-b9e5-67cc51154382 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:29.343333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 447/875] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:37:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:29.349980 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f06e1-edff-4b02-951c-905b97625a78 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:29.465728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f06e1-edff-4b02-951c-905b97625a78 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:29.481897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:29.510123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0aee9a-9dd9-4d4b-ae01-ce5d7b5c53a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:29.510760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0aee9a-9dd9-4d4b-ae01-ce5d7b5c53a2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:29.511039 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0aee9a-9dd9-4d4b-ae01-ce5d7b5c53a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:29.511306 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0aee9a-9dd9-4d4b-ae01-ce5d7b5c53a2 None admin] PATH_INFO: `/v3/users/82751009ca4a4cb99090d3d96520305d` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:29.512464 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0aee9a-9dd9-4d4b-ae01-ce5d7b5c53a2 None admin] RBAC: Authorizing `identity:get_user(user_id=82751009ca4a4cb99090d3d96520305d)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:29.513903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0aee9a-9dd9-4d4b-ae01-ce5d7b5c53a2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:29.515499 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 429/876] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:37:29 2022] GET /identity/v3/users/82751009ca4a4cb99090d3d96520305d => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:29.522053 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf38499-6380-424d-b9e5-67cc51154382 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:29.543469 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:29.637940 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf38499-6380-424d-b9e5-67cc51154382 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:29.677143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e3a5a-28fa-4cd2-81c8-41ced3604331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:29.677898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e3a5a-28fa-4cd2-81c8-41ced3604331 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:29.678158 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e3a5a-28fa-4cd2-81c8-41ced3604331 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:29.678290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e3a5a-28fa-4cd2-81c8-41ced3604331 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:29.755409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e3a5a-28fa-4cd2-81c8-41ced3604331 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:29.758358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e3a5a-28fa-4cd2-81c8-41ced3604331 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:29.780206 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:29.872202 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 448/877] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:37:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 13:37:29.878772 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0aee9a-9dd9-4d4b-ae01-ce5d7b5c53a2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:30.118870 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0aee9a-9dd9-4d4b-ae01-ce5d7b5c53a2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:30.159208 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768a142-e988-4e11-aaff-8e7f9bfc8d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:30.159768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768a142-e988-4e11-aaff-8e7f9bfc8d72 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:30.160073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768a142-e988-4e11-aaff-8e7f9bfc8d72 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:30.160337 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768a142-e988-4e11-aaff-8e7f9bfc8d72 None admin] PATH_INFO: `/v3/projects/a532c3269a284e9587b5601358a40c93` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:30.161669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768a142-e988-4e11-aaff-8e7f9bfc8d72 None admin] RBAC: Authorizing `identity:get_project(project_id=a532c3269a284e9587b5601358a40c93)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:30.163208 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768a142-e988-4e11-aaff-8e7f9bfc8d72 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:30.164992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 430/878] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:37:29 2022] GET /identity/v3/projects/a532c3269a284e9587b5601358a40c93 => generated 311 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:30.172349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e3a5a-28fa-4cd2-81c8-41ced3604331 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:30.278461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e3a5a-28fa-4cd2-81c8-41ced3604331 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:30.297559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:30.315609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c20594-d48d-4c66-96e2-a54002d5daad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:30.316241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c20594-d48d-4c66-96e2-a54002d5daad None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:30.316516 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c20594-d48d-4c66-96e2-a54002d5daad None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:30.316775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c20594-d48d-4c66-96e2-a54002d5daad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:30.317696 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c20594-d48d-4c66-96e2-a54002d5daad None admin] RBAC: Adding query filter params (role.id=57740dc2ebe1429294f20dd51592b82a, scope.project.id=a532c3269a284e9587b5601358a40c93, user.id=82751009ca4a4cb99090d3d96520305d) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:30.318019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c20594-d48d-4c66-96e2-a54002d5daad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:30.319616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c20594-d48d-4c66-96e2-a54002d5daad None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:30.326441 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 449/879] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:37:30 2022] GET /identity/v3/role_assignments?user.id=82751009ca4a4cb99090d3d96520305d&scope.project.id=a532c3269a284e9587b5601358a40c93&role.id=57740dc2ebe1429294f20dd51592b82a => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:32.571461 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6a926-68be-442c-a629-9438b4a5dbdb None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:32.572026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6a926-68be-442c-a629-9438b4a5dbdb None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:32.572026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6a926-68be-442c-a629-9438b4a5dbdb None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:32.573469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 431/880] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:32.579222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275d2c3-82e2-4a46-b5ad-4ae708bd4811 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:32.579838 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275d2c3-82e2-4a46-b5ad-4ae708bd4811 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:32.579838 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275d2c3-82e2-4a46-b5ad-4ae708bd4811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:32.598767 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-6275d2c3-82e2-4a46-b5ad-4ae708bd4811 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:32.600022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6275d2c3-82e2-4a46-b5ad-4ae708bd4811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:32.671899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 450/881] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:32.942473 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6a926-68be-442c-a629-9438b4a5dbdb None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:33.001199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:33.054974 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6a926-68be-442c-a629-9438b4a5dbdb None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:33.091554 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f05d64-1761-4f9a-99e6-47035dee352d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:33.092191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f05d64-1761-4f9a-99e6-47035dee352d None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:33.092465 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f05d64-1761-4f9a-99e6-47035dee352d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:33.092735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f05d64-1761-4f9a-99e6-47035dee352d None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:33.093398 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f05d64-1761-4f9a-99e6-47035dee352d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:33.094807 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f05d64-1761-4f9a-99e6-47035dee352d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:33.122973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 432/882] 10.222.0.36 () {60 vars in 1138 bytes} [Sat Oct 15 13:37:32 2022] POST /identity/v3/domains => generated 267 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:37:35.326471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558359-8a71-41c9-845b-1fb0f406112d None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:35.326998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558359-8a71-41c9-845b-1fb0f406112d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:35.326998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558359-8a71-41c9-845b-1fb0f406112d None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:35.328360 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 451/883] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:35.334360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54715531-f6ae-4c5b-bfd6-bdd91119310c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:35.334837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54715531-f6ae-4c5b-bfd6-bdd91119310c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:35.334926 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54715531-f6ae-4c5b-bfd6-bdd91119310c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:35.355232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-54715531-f6ae-4c5b-bfd6-bdd91119310c None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:35.356468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-54715531-f6ae-4c5b-bfd6-bdd91119310c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:35.436559 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 433/884] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:37:35.728804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6558359-8a71-41c9-845b-1fb0f406112d None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:35.841429 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6558359-8a71-41c9-845b-1fb0f406112d None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:35.855821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:35.878532 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8046a9-57bb-4dd4-80fb-782b3ba3421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:35.879164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8046a9-57bb-4dd4-80fb-782b3ba3421d None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:35.879457 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8046a9-57bb-4dd4-80fb-782b3ba3421d None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:35.879729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8046a9-57bb-4dd4-80fb-782b3ba3421d None admin] PATH_INFO: `/v3/domains/411b0eda67c34e988b77e2eb94d81e7a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:35.891526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8046a9-57bb-4dd4-80fb-782b3ba3421d None admin] RBAC: Authorizing `identity:get_domain(domain_id=411b0eda67c34e988b77e2eb94d81e7a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:35.893569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8046a9-57bb-4dd4-80fb-782b3ba3421d None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:35.895461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 452/885] 10.222.0.36 () {56 vars in 1183 bytes} [Sat Oct 15 13:37:35 2022] GET /identity/v3/domains/411b0eda67c34e988b77e2eb94d81e7a => generated 267 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:35.902274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54715531-f6ae-4c5b-bfd6-bdd91119310c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:35.947814 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:36.009909 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54715531-f6ae-4c5b-bfd6-bdd91119310c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:36.046700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ecab13-24be-4ed8-8abc-5021aa75a6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:36.047369 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecab13-24be-4ed8-8abc-5021aa75a6db None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:36.047643 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecab13-24be-4ed8-8abc-5021aa75a6db None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:36.047993 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecab13-24be-4ed8-8abc-5021aa75a6db None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:36.048761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ecab13-24be-4ed8-8abc-5021aa75a6db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:36.050239 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ecab13-24be-4ed8-8abc-5021aa75a6db None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:36.078744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 434/886] 10.222.0.36 () {60 vars in 1141 bytes} [Sat Oct 15 13:37:35 2022] POST /identity/v3/projects => generated 361 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:37:38.283968 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5da85-a099-4a8f-a312-2ea343b76a9a None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:38.284493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5da85-a099-4a8f-a312-2ea343b76a9a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:38.284493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5da85-a099-4a8f-a312-2ea343b76a9a None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:38.286238 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 453/887] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:38.291923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae0ab4-4350-428e-9e36-0129049d307b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:38.292328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae0ab4-4350-428e-9e36-0129049d307b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:38.292459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae0ab4-4350-428e-9e36-0129049d307b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:38.311669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5eae0ab4-4350-428e-9e36-0129049d307b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:38.312902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5eae0ab4-4350-428e-9e36-0129049d307b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:38.383503 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 435/888] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:38.651908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5da85-a099-4a8f-a312-2ea343b76a9a None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:38.755203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5da85-a099-4a8f-a312-2ea343b76a9a None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:38.790879 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89c701-03f7-4e8c-8901-80c8d3390343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:38.791501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89c701-03f7-4e8c-8901-80c8d3390343 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:38.791775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89c701-03f7-4e8c-8901-80c8d3390343 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:38.792075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89c701-03f7-4e8c-8901-80c8d3390343 None admin] PATH_INFO: `/v3/domains/411b0eda67c34e988b77e2eb94d81e7a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:38.793231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89c701-03f7-4e8c-8901-80c8d3390343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=411b0eda67c34e988b77e2eb94d81e7a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:38.794646 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89c701-03f7-4e8c-8901-80c8d3390343 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:38.796228 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 454/889] 10.222.0.36 () {56 vars in 1183 bytes} [Sat Oct 15 13:37:38 2022] GET /identity/v3/domains/411b0eda67c34e988b77e2eb94d81e7a => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:38.802548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae0ab4-4350-428e-9e36-0129049d307b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:38.826858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:38.906219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae0ab4-4350-428e-9e36-0129049d307b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:38.943586 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341a927-c0c0-4d25-be3a-fec754e69f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:38.944243 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341a927-c0c0-4d25-be3a-fec754e69f6d None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:38.944515 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341a927-c0c0-4d25-be3a-fec754e69f6d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:38.944782 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341a927-c0c0-4d25-be3a-fec754e69f6d None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:38.945428 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341a927-c0c0-4d25-be3a-fec754e69f6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:38.946875 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341a927-c0c0-4d25-be3a-fec754e69f6d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:38.969099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 436/890] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:37:38 2022] POST /identity/v3/users => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:37:41.155054 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a610a9c-ce54-43cf-a72a-08cca4bad266 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:41.155641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a610a9c-ce54-43cf-a72a-08cca4bad266 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:41.155641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a610a9c-ce54-43cf-a72a-08cca4bad266 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:41.156879 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 455/891] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:41.162703 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0b83-b170-4aa1-8da4-ad45133214bb None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:41.163178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0b83-b170-4aa1-8da4-ad45133214bb None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:41.163280 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0b83-b170-4aa1-8da4-ad45133214bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:41.183688 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-773b0b83-b170-4aa1-8da4-ad45133214bb None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:41.184926 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-773b0b83-b170-4aa1-8da4-ad45133214bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:41.259759 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 437/892] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:37:41.525999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a610a9c-ce54-43cf-a72a-08cca4bad266 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:41.572378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:41.632740 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a610a9c-ce54-43cf-a72a-08cca4bad266 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:41.666506 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a0549-f8cf-40a2-8245-762c2e07429d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:41.667123 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a0549-f8cf-40a2-8245-762c2e07429d None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:41.667393 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a0549-f8cf-40a2-8245-762c2e07429d None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:41.667659 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a0549-f8cf-40a2-8245-762c2e07429d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:41.678080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a0549-f8cf-40a2-8245-762c2e07429d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-6e4a0549-f8cf-40a2-8245-762c2e07429d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:37:41.682762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:41.683845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:41.684748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:41.684748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:41.684748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:41.684748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:41.684748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:37:41.684748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:37:41.684748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:37:41.684748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:37:41.684748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:37:41.684748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:37:41.684748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:37:41.684748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:37:41.684748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 456/893] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:37:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:37:41.690386 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b0b83-b170-4aa1-8da4-ad45133214bb None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:41.700447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:41.793596 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b0b83-b170-4aa1-8da4-ad45133214bb None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:41.832094 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135966e-de9d-4012-9d4b-6121545f68cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:41.832710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135966e-de9d-4012-9d4b-6121545f68cd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:41.832977 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135966e-de9d-4012-9d4b-6121545f68cd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:41.833238 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135966e-de9d-4012-9d4b-6121545f68cd None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:41.833894 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135966e-de9d-4012-9d4b-6121545f68cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:41.834186 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135966e-de9d-4012-9d4b-6121545f68cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:41.835552 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135966e-de9d-4012-9d4b-6121545f68cd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:41.844344 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 438/894] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:37:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:41.850770 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a0549-f8cf-40a2-8245-762c2e07429d None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:41.951657 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:41.955315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a0549-f8cf-40a2-8245-762c2e07429d None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:41.992394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9cd83-d34c-4428-8299-f5fbeb8c024a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:41.993016 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9cd83-d34c-4428-8299-f5fbeb8c024a None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:41.993289 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9cd83-d34c-4428-8299-f5fbeb8c024a None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:41.993540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9cd83-d34c-4428-8299-f5fbeb8c024a None admin] PATH_INFO: `/v3/users/6c36465dd572410ca195d4b9baf12c10` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:42.007988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9cd83-d34c-4428-8299-f5fbeb8c024a None admin] RBAC: Authorizing `identity:get_user(user_id=6c36465dd572410ca195d4b9baf12c10)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:42.009428 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9cd83-d34c-4428-8299-f5fbeb8c024a None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:42.011071 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 457/895] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:37:41 2022] GET /identity/v3/users/6c36465dd572410ca195d4b9baf12c10 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:42.017425 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135966e-de9d-4012-9d4b-6121545f68cd None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:42.087719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:42.122508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135966e-de9d-4012-9d4b-6121545f68cd None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:42.157966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74645a4-4487-4e31-89bf-dbc3ba0a4fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:42.158685 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74645a4-4487-4e31-89bf-dbc3ba0a4fa8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:42.158961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74645a4-4487-4e31-89bf-dbc3ba0a4fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:42.159230 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74645a4-4487-4e31-89bf-dbc3ba0a4fa8 None admin] PATH_INFO: `/v3/projects/3cb26c93ebcb4b7abe7d6fca2b805145` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:42.160418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74645a4-4487-4e31-89bf-dbc3ba0a4fa8 None admin] RBAC: Authorizing `identity:get_project(project_id=3cb26c93ebcb4b7abe7d6fca2b805145)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:42.161850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74645a4-4487-4e31-89bf-dbc3ba0a4fa8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:42.163469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 439/896] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:37:42 2022] GET /identity/v3/projects/3cb26c93ebcb4b7abe7d6fca2b805145 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:42.169805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9cd83-d34c-4428-8299-f5fbeb8c024a None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:42.272099 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9cd83-d34c-4428-8299-f5fbeb8c024a None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:42.306314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289845c-88ea-4619-a393-8b7e2ccc4d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:42.306959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289845c-88ea-4619-a393-8b7e2ccc4d87 None admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:42.307229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289845c-88ea-4619-a393-8b7e2ccc4d87 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:42.307497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289845c-88ea-4619-a393-8b7e2ccc4d87 None admin] PATH_INFO: `/v3/projects/3cb26c93ebcb4b7abe7d6fca2b805145/users/6c36465dd572410ca195d4b9baf12c10/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:42.309520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289845c-88ea-4619-a393-8b7e2ccc4d87 None admin] RBAC: Authorizing `identity:create_grant(project_id=3cb26c93ebcb4b7abe7d6fca2b805145, user_id=6c36465dd572410ca195d4b9baf12c10, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:42.310940 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289845c-88ea-4619-a393-8b7e2ccc4d87 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:42.325385 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 458/897] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:37:42 2022] PUT /identity/v3/projects/3cb26c93ebcb4b7abe7d6fca2b805145/users/6c36465dd572410ca195d4b9baf12c10/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:37:44.417532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753de27-9990-45ae-8b6b-661ccc2cadb0 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:44.418100 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753de27-9990-45ae-8b6b-661ccc2cadb0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:44.418100 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753de27-9990-45ae-8b6b-661ccc2cadb0 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:44.419426 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 440/898] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:37:44.425540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795373c5-0eb9-47f9-804f-b0eac18b7082 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:44.426015 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795373c5-0eb9-47f9-804f-b0eac18b7082 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:44.426015 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795373c5-0eb9-47f9-804f-b0eac18b7082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:44.444605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-795373c5-0eb9-47f9-804f-b0eac18b7082 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:37:44.445914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-795373c5-0eb9-47f9-804f-b0eac18b7082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:37:44.459912 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:44.517178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 459/899] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:37:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:37:44.789753 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753de27-9990-45ae-8b6b-661ccc2cadb0 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:44.892534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753de27-9990-45ae-8b6b-661ccc2cadb0 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:44.928942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68db542-7265-4d3b-ac31-caf80817ca9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:44.929599 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68db542-7265-4d3b-ac31-caf80817ca9d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:44.929872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68db542-7265-4d3b-ac31-caf80817ca9d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:44.930139 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68db542-7265-4d3b-ac31-caf80817ca9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:44.938857 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:44.944193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68db542-7265-4d3b-ac31-caf80817ca9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-c68db542-7265-4d3b-ac31-caf80817ca9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:37:44.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:37:44.950644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:37:44.951480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:37:44.951480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:37:44.951480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:37:44.951480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:37:44.951480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:37:44.951480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:37:44.951480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:37:44.951480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:37:44.951480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:37:44.951480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:37:44.951480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:37:44.951480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:37:44.951480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 441/900] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:37:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:37:44.957365 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795373c5-0eb9-47f9-804f-b0eac18b7082 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:45.059972 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795373c5-0eb9-47f9-804f-b0eac18b7082 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:45.094349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf562b49-0609-4ca4-a83e-f442af4e2066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:45.095043 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf562b49-0609-4ca4-a83e-f442af4e2066 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:45.095323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf562b49-0609-4ca4-a83e-f442af4e2066 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:45.095589 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf562b49-0609-4ca4-a83e-f442af4e2066 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:45.096277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf562b49-0609-4ca4-a83e-f442af4e2066 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:45.096573 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf562b49-0609-4ca4-a83e-f442af4e2066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:45.097928 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf562b49-0609-4ca4-a83e-f442af4e2066 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:45.106038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 460/901] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:37:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:45.113042 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68db542-7265-4d3b-ac31-caf80817ca9d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:45.217319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68db542-7265-4d3b-ac31-caf80817ca9d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:45.252973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095be8b-11ae-490b-84bb-89373c9e65ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:45.253601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095be8b-11ae-490b-84bb-89373c9e65ea None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:45.253876 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095be8b-11ae-490b-84bb-89373c9e65ea None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:45.254145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095be8b-11ae-490b-84bb-89373c9e65ea None admin] PATH_INFO: `/v3/users/6c36465dd572410ca195d4b9baf12c10` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:45.255306 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095be8b-11ae-490b-84bb-89373c9e65ea None admin] RBAC: Authorizing `identity:get_user(user_id=6c36465dd572410ca195d4b9baf12c10)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:45.256785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095be8b-11ae-490b-84bb-89373c9e65ea None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:45.258694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 442/902] 10.222.0.36 () {56 vars in 1177 bytes} [Sat Oct 15 13:37:45 2022] GET /identity/v3/users/6c36465dd572410ca195d4b9baf12c10 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:45.264847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf562b49-0609-4ca4-a83e-f442af4e2066 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:45.278168 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:45.370827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf562b49-0609-4ca4-a83e-f442af4e2066 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:45.406908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc3de4-f440-49b6-8e5f-5f5a5ee1b90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:45.407562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc3de4-f440-49b6-8e5f-5f5a5ee1b90f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:45.407910 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc3de4-f440-49b6-8e5f-5f5a5ee1b90f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:45.408260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc3de4-f440-49b6-8e5f-5f5a5ee1b90f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:45.476055 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc3de4-f440-49b6-8e5f-5f5a5ee1b90f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:45.477502 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc3de4-f440-49b6-8e5f-5f5a5ee1b90f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:45.488978 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:45.582468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 461/903] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:37:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 13:37:45.588829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095be8b-11ae-490b-84bb-89373c9e65ea None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:45.657842 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:45.698703 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095be8b-11ae-490b-84bb-89373c9e65ea None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:45.734664 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0760337-9f27-4e68-b5fb-103d4578f198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:45.735296 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0760337-9f27-4e68-b5fb-103d4578f198 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:45.735595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0760337-9f27-4e68-b5fb-103d4578f198 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:45.735866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0760337-9f27-4e68-b5fb-103d4578f198 None admin] PATH_INFO: `/v3/projects/3cb26c93ebcb4b7abe7d6fca2b805145` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:45.737062 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0760337-9f27-4e68-b5fb-103d4578f198 None admin] RBAC: Authorizing `identity:get_project(project_id=3cb26c93ebcb4b7abe7d6fca2b805145)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:45.738514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0760337-9f27-4e68-b5fb-103d4578f198 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:45.740181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 443/904] 10.222.0.36 () {56 vars in 1186 bytes} [Sat Oct 15 13:37:45 2022] GET /identity/v3/projects/3cb26c93ebcb4b7abe7d6fca2b805145 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:45.746717 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc3de4-f440-49b6-8e5f-5f5a5ee1b90f None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:37:45.849957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc3de4-f440-49b6-8e5f-5f5a5ee1b90f None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:37:45.861355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:45.886861 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d580ba-fa88-4d5b-9066-fbbe10505b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:37:45.887465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d580ba-fa88-4d5b-9066-fbbe10505b7e None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:37:45.887735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d580ba-fa88-4d5b-9066-fbbe10505b7e None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:37:45.888034 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d580ba-fa88-4d5b-9066-fbbe10505b7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:37:45.889160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d580ba-fa88-4d5b-9066-fbbe10505b7e None admin] RBAC: Adding query filter params (role.id=57740dc2ebe1429294f20dd51592b82a, scope.project.id=3cb26c93ebcb4b7abe7d6fca2b805145, user.id=6c36465dd572410ca195d4b9baf12c10) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:37:45.889160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d580ba-fa88-4d5b-9066-fbbe10505b7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:37:45.890442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d580ba-fa88-4d5b-9066-fbbe10505b7e None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:37:45.896591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 462/905] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:37:45 2022] GET /identity/v3/role_assignments?user.id=6c36465dd572410ca195d4b9baf12c10&scope.project.id=3cb26c93ebcb4b7abe7d6fca2b805145&role.id=57740dc2ebe1429294f20dd51592b82a => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:37:55.729402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=33 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:37:55.893715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=13 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:44.201412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d9b1c-daf3-4030-ae11-9911c2949c8f None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:44.202563 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d9b1c-daf3-4030-ae11-9911c2949c8f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:44.202800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d9b1c-daf3-4030-ae11-9911c2949c8f None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:44.204449 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 444/906] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:38:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:38:44.211263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9997db-ea80-4009-b09f-e0eede6179be None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:44.212151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9997db-ea80-4009-b09f-e0eede6179be None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:44.212151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9997db-ea80-4009-b09f-e0eede6179be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:44.234450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-da9997db-ea80-4009-b09f-e0eede6179be None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:38:44.236466 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-da9997db-ea80-4009-b09f-e0eede6179be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:38:44.338581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 463/907] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:38:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:38:44.617301 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d9b1c-daf3-4030-ae11-9911c2949c8f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:44.774191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d9b1c-daf3-4030-ae11-9911c2949c8f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:44.824522 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7c94a-49df-4e6e-b207-16652561ce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:44.825828 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7c94a-49df-4e6e-b207-16652561ce9c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:44.826426 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7c94a-49df-4e6e-b207-16652561ce9c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:44.826905 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7c94a-49df-4e6e-b207-16652561ce9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:44.849290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7c94a-49df-4e6e-b207-16652561ce9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:44.851258 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7c94a-49df-4e6e-b207-16652561ce9c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-0fa7c94a-49df-4e6e-b207-16652561ce9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:38:44.868360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:44.869694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:38:44.870649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:38:44.870649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:38:44.870649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:38:44.870649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 445/908] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:38:44.877400 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9997db-ea80-4009-b09f-e0eede6179be None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:45.067124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9997db-ea80-4009-b09f-e0eede6179be None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:45.130674 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:45.141449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70864c6e-7748-4b97-a34f-be78a13c377d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:45.142153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70864c6e-7748-4b97-a34f-be78a13c377d None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:45.142479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70864c6e-7748-4b97-a34f-be78a13c377d None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:45.142839 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70864c6e-7748-4b97-a34f-be78a13c377d None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:45.143677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70864c6e-7748-4b97-a34f-be78a13c377d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:45.144140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70864c6e-7748-4b97-a34f-be78a13c377d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:45.146048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70864c6e-7748-4b97-a34f-be78a13c377d None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:45.167513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 464/909] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:38:45.169959 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7c94a-49df-4e6e-b207-16652561ce9c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:45.259223 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:45.331457 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7c94a-49df-4e6e-b207-16652561ce9c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:45.373082 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124f6e8-2dff-4ed5-af2e-2af9c3cf1455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:45.373721 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124f6e8-2dff-4ed5-af2e-2af9c3cf1455 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:45.373989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124f6e8-2dff-4ed5-af2e-2af9c3cf1455 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:45.374258 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124f6e8-2dff-4ed5-af2e-2af9c3cf1455 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:45.374937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124f6e8-2dff-4ed5-af2e-2af9c3cf1455 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:45.376507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124f6e8-2dff-4ed5-af2e-2af9c3cf1455 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:45.406841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 446/910] 10.222.0.36 () {60 vars in 1132 bytes} [Sat Oct 15 13:38:45 2022] POST /identity/v3/users => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:38:47.897198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d2e9c-f620-4adc-8d0e-048a4f06f7be None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:47.897702 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d2e9c-f620-4adc-8d0e-048a4f06f7be None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:47.897801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d2e9c-f620-4adc-8d0e-048a4f06f7be None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:47.899275 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 465/911] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:38:47.904678 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001420c2-bc67-406b-84a6-7239b4b57120 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:47.905126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001420c2-bc67-406b-84a6-7239b4b57120 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:47.905218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001420c2-bc67-406b-84a6-7239b4b57120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:47.925282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-001420c2-bc67-406b-84a6-7239b4b57120 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:38:47.926575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-001420c2-bc67-406b-84a6-7239b4b57120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:38:47.998794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 447/912] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:38:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:38:48.274867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d2e9c-f620-4adc-8d0e-048a4f06f7be None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:48.397726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d2e9c-f620-4adc-8d0e-048a4f06f7be None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:48.440494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b381f6-150c-4ba6-aeab-1e6c2d7a8aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:48.441194 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b381f6-150c-4ba6-aeab-1e6c2d7a8aa3 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:48.441521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b381f6-150c-4ba6-aeab-1e6c2d7a8aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:48.441795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b381f6-150c-4ba6-aeab-1e6c2d7a8aa3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:48.454503 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b381f6-150c-4ba6-aeab-1e6c2d7a8aa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-a0b381f6-150c-4ba6-aeab-1e6c2d7a8aa3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:38:48.460638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:48.461994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:48.462828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:48.462828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:48.462828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:48.462828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:48.462828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:38:48.462828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:38:48.462828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:38:48.462828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:38:48.462828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:38:48.462828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:38:48.462828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:38:48.462828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:38:48.462828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 466/913] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:38:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:38:48.470477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001420c2-bc67-406b-84a6-7239b4b57120 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:48.514593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:48.585153 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001420c2-bc67-406b-84a6-7239b4b57120 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:48.627828 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9706fa-eb09-4ab8-9175-79838a0d8f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:48.628479 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9706fa-eb09-4ab8-9175-79838a0d8f17 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:48.628774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9706fa-eb09-4ab8-9175-79838a0d8f17 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:48.629014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9706fa-eb09-4ab8-9175-79838a0d8f17 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:48.629716 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9706fa-eb09-4ab8-9175-79838a0d8f17 None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:48.630015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9706fa-eb09-4ab8-9175-79838a0d8f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:48.631477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9706fa-eb09-4ab8-9175-79838a0d8f17 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:48.641333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 448/914] 10.222.0.36 () {56 vars in 1103 bytes} [Sat Oct 15 13:38:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:38:48.647594 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b381f6-150c-4ba6-aeab-1e6c2d7a8aa3 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:48.711274 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:48.775172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b381f6-150c-4ba6-aeab-1e6c2d7a8aa3 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:48.815904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365ad1e-194b-40e8-8746-1f0534ae7fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:48.816619 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365ad1e-194b-40e8-8746-1f0534ae7fc8 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:48.816921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365ad1e-194b-40e8-8746-1f0534ae7fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:48.817148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365ad1e-194b-40e8-8746-1f0534ae7fc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:48.830526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365ad1e-194b-40e8-8746-1f0534ae7fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:48.832114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365ad1e-194b-40e8-8746-1f0534ae7fc8 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-4365ad1e-194b-40e8-8746-1f0534ae7fc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:38:48.849762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:48.850801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:38:48.856614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:38:48.856614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:38:48.856614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:38:48.856614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 467/915] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:38:48.856860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9706fa-eb09-4ab8-9175-79838a0d8f17 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:48.990259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:48.994835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9706fa-eb09-4ab8-9175-79838a0d8f17 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:49.033497 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694158ae-3ffb-4827-bda6-975d1178adcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:49.034181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694158ae-3ffb-4827-bda6-975d1178adcc None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:49.034499 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694158ae-3ffb-4827-bda6-975d1178adcc None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:49.034761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694158ae-3ffb-4827-bda6-975d1178adcc None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:49.035409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694158ae-3ffb-4827-bda6-975d1178adcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:49.038733 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694158ae-3ffb-4827-bda6-975d1178adcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:49.038733 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694158ae-3ffb-4827-bda6-975d1178adcc None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:49.050871 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 449/916] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:38:49.057203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365ad1e-194b-40e8-8746-1f0534ae7fc8 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:49.168640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365ad1e-194b-40e8-8746-1f0534ae7fc8 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:49.170359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:49.210378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50abf462-2520-40a1-864e-01194e83f4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:49.211066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abf462-2520-40a1-864e-01194e83f4f6 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:49.211386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abf462-2520-40a1-864e-01194e83f4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:49.213142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abf462-2520-40a1-864e-01194e83f4f6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:49.220911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50abf462-2520-40a1-864e-01194e83f4f6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:49.222463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50abf462-2520-40a1-864e-01194e83f4f6 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-50abf462-2520-40a1-864e-01194e83f4f6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:38:49.240550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:49.242066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:49.243075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:49.243075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:49.243075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:38:49.243075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:38:49.243075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:38:49.243075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:38:49.243075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:38:49.243075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:38:49.243075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 13:38:49.243075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:38:49.243075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 468/917] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:38:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:38:49.248577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694158ae-3ffb-4827-bda6-975d1178adcc None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:49.412446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694158ae-3ffb-4827-bda6-975d1178adcc None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:49.460041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de67f8-322a-4b0f-9a71-c8e7b37396fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:49.460041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de67f8-322a-4b0f-9a71-c8e7b37396fe None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:49.460041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de67f8-322a-4b0f-9a71-c8e7b37396fe None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:49.460041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de67f8-322a-4b0f-9a71-c8e7b37396fe None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:49.460755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de67f8-322a-4b0f-9a71-c8e7b37396fe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:49.460755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de67f8-322a-4b0f-9a71-c8e7b37396fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:49.460755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de67f8-322a-4b0f-9a71-c8e7b37396fe None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:49.488009 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 450/918] 10.222.0.36 () {56 vars in 1143 bytes} [Sat Oct 15 13:38:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:38:49.494409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50abf462-2520-40a1-864e-01194e83f4f6 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:49.622636 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50abf462-2520-40a1-864e-01194e83f4f6 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:49.676039 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596142d-20d6-4a4d-8b87-7ea052ad4d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:49.676726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596142d-20d6-4a4d-8b87-7ea052ad4d63 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:49.677026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596142d-20d6-4a4d-8b87-7ea052ad4d63 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:49.677294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596142d-20d6-4a4d-8b87-7ea052ad4d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:49.689239 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596142d-20d6-4a4d-8b87-7ea052ad4d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:49.691531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596142d-20d6-4a4d-8b87-7ea052ad4d63 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-7596142d-20d6-4a4d-8b87-7ea052ad4d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:38:49.708164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:49.709204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:38:49.710760 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:38:49.710760 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:38:49.710760 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:38:49.710760 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 469/919] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:38:49.715443 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de67f8-322a-4b0f-9a71-c8e7b37396fe None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:49.739914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:49.850839 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de67f8-322a-4b0f-9a71-c8e7b37396fe None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:49.922482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51223568-49f9-42c0-a69f-1411bb12dbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:49.923402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51223568-49f9-42c0-a69f-1411bb12dbcd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:49.923866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51223568-49f9-42c0-a69f-1411bb12dbcd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:49.924548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51223568-49f9-42c0-a69f-1411bb12dbcd None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:49.926098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51223568-49f9-42c0-a69f-1411bb12dbcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:49.926570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51223568-49f9-42c0-a69f-1411bb12dbcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:49.928972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51223568-49f9-42c0-a69f-1411bb12dbcd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:49.948765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 451/920] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:38:49.954715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596142d-20d6-4a4d-8b87-7ea052ad4d63 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:49.972427 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:50.087969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596142d-20d6-4a4d-8b87-7ea052ad4d63 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:50.126429 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8e379-bcb1-4d7e-9083-f15e183b609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:50.127202 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8e379-bcb1-4d7e-9083-f15e183b609b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:50.127508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8e379-bcb1-4d7e-9083-f15e183b609b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:50.127840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8e379-bcb1-4d7e-9083-f15e183b609b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:50.135827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8e379-bcb1-4d7e-9083-f15e183b609b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:50.137450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8e379-bcb1-4d7e-9083-f15e183b609b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-a6b8e379-bcb1-4d7e-9083-f15e183b609b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:38:50.151830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:50.152925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:50.153813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:38:50.153813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:38:50.153813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:38:50.153813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:38:50.153813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:38:50.153813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:38:50.153813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:38:50.153813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:38:50.153813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 470/921] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:38:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:38:50.160010 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51223568-49f9-42c0-a69f-1411bb12dbcd None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:50.256570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:50.281323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51223568-49f9-42c0-a69f-1411bb12dbcd None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:50.321433 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65e1a0-ce1b-4d02-a69b-5b911e5ffb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:50.322344 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65e1a0-ce1b-4d02-a69b-5b911e5ffb1f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:50.322679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65e1a0-ce1b-4d02-a69b-5b911e5ffb1f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:50.322947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65e1a0-ce1b-4d02-a69b-5b911e5ffb1f None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:50.324047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65e1a0-ce1b-4d02-a69b-5b911e5ffb1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:50.324552 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65e1a0-ce1b-4d02-a69b-5b911e5ffb1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:50.326942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65e1a0-ce1b-4d02-a69b-5b911e5ffb1f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:50.341518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 452/922] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:38:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:38:50.347759 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8e379-bcb1-4d7e-9083-f15e183b609b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:50.437293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:50.467853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8e379-bcb1-4d7e-9083-f15e183b609b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:50.506035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18938e69-d355-4305-a585-1cc7cf3ff9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:50.506758 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18938e69-d355-4305-a585-1cc7cf3ff9eb None admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:50.506981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18938e69-d355-4305-a585-1cc7cf3ff9eb None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:50.507249 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18938e69-d355-4305-a585-1cc7cf3ff9eb None admin] PATH_INFO: `/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/1c6720518c03413aa8f85e4d6142fe87/roles/8b837ae2ed57479a93acb638b80e2d67` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:50.523244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18938e69-d355-4305-a585-1cc7cf3ff9eb None admin] RBAC: Authorizing `identity:create_grant(project_id=2a5a64cc8aa04972bb464407cb0312b4, user_id=1c6720518c03413aa8f85e4d6142fe87, role_id=8b837ae2ed57479a93acb638b80e2d67)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:50.524732 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18938e69-d355-4305-a585-1cc7cf3ff9eb None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:50.540473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 471/923] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:38:50 2022] PUT /identity/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/1c6720518c03413aa8f85e4d6142fe87/roles/8b837ae2ed57479a93acb638b80e2d67 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:38:52.916306 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcac00-6171-4110-ad0b-656d19ddf134 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:52.916887 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcac00-6171-4110-ad0b-656d19ddf134 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:52.917012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcac00-6171-4110-ad0b-656d19ddf134 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:52.918459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 453/924] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:38:52.925322 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa8541-0dce-487b-9f43-241358cf823b None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:52.925900 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa8541-0dce-487b-9f43-241358cf823b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:52.925900 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa8541-0dce-487b-9f43-241358cf823b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:52.945792 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-c8aa8541-0dce-487b-9f43-241358cf823b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:38:52.947326 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c8aa8541-0dce-487b-9f43-241358cf823b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:38:53.031374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 472/925] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:38:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:38:53.312664 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcac00-6171-4110-ad0b-656d19ddf134 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:53.425613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcac00-6171-4110-ad0b-656d19ddf134 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:53.464715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3cb900-64e2-4ef6-be99-8a1d1acf249c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:53.465545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3cb900-64e2-4ef6-be99-8a1d1acf249c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:53.465877 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3cb900-64e2-4ef6-be99-8a1d1acf249c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:53.466167 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3cb900-64e2-4ef6-be99-8a1d1acf249c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:53.476412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:53.481635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3cb900-64e2-4ef6-be99-8a1d1acf249c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-4e3cb900-64e2-4ef6-be99-8a1d1acf249c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:38:53.487362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:53.488381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:53.489236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:53.489236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:53.489236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:53.489236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:53.489236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:38:53.489236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:38:53.489236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:38:53.489236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:38:53.489236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:38:53.489236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:38:53.489236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 15 13:38:53.489236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:38:53.489236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 454/926] 10.222.0.36 () {56 vars in 1102 bytes} [Sat Oct 15 13:38:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:38:53.495409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa8541-0dce-487b-9f43-241358cf823b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:53.553473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:53.604399 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa8541-0dce-487b-9f43-241358cf823b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:53.641859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99212398-52fa-48e6-a292-b216d961f72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:53.642480 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99212398-52fa-48e6-a292-b216d961f72b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:53.642860 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99212398-52fa-48e6-a292-b216d961f72b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:53.643132 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99212398-52fa-48e6-a292-b216d961f72b None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:53.644102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99212398-52fa-48e6-a292-b216d961f72b None admin] RBAC: Adding query filter params (name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:53.645093 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99212398-52fa-48e6-a292-b216d961f72b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:53.645811 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99212398-52fa-48e6-a292-b216d961f72b None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:53.654314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 473/927] 10.222.0.36 () {56 vars in 1103 bytes} [Sat Oct 15 13:38:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:38:53.661300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3cb900-64e2-4ef6-be99-8a1d1acf249c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:53.776655 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3cb900-64e2-4ef6-be99-8a1d1acf249c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:53.814158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8c340-9da3-4e88-b915-a3346aaa51cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:53.814818 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8c340-9da3-4e88-b915-a3346aaa51cd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:53.815095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8c340-9da3-4e88-b915-a3346aaa51cd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:53.815460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8c340-9da3-4e88-b915-a3346aaa51cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:53.825597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8c340-9da3-4e88-b915-a3346aaa51cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:53.827108 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8c340-9da3-4e88-b915-a3346aaa51cd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-85f8c340-9da3-4e88-b915-a3346aaa51cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:38:53.844245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:53.845329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:38:53.847024 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:38:53.847024 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:38:53.847024 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:38:53.847024 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 455/928] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:38:53.852087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99212398-52fa-48e6-a292-b216d961f72b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:53.972968 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99212398-52fa-48e6-a292-b216d961f72b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:53.991984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:54.012173 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c724fc-6fa8-485f-88f6-e45647ba68af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:54.012798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c724fc-6fa8-485f-88f6-e45647ba68af None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:54.013074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c724fc-6fa8-485f-88f6-e45647ba68af None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:54.013342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c724fc-6fa8-485f-88f6-e45647ba68af None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:54.014044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c724fc-6fa8-485f-88f6-e45647ba68af None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:54.014347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c724fc-6fa8-485f-88f6-e45647ba68af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:54.015999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c724fc-6fa8-485f-88f6-e45647ba68af None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:54.029535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 474/929] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:38:54.036031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8c340-9da3-4e88-b915-a3346aaa51cd None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:54.090647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:54.145793 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8c340-9da3-4e88-b915-a3346aaa51cd None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:54.182132 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b51be-9994-450e-bda7-a63e3243fe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:54.182773 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b51be-9994-450e-bda7-a63e3243fe1a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:54.183051 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b51be-9994-450e-bda7-a63e3243fe1a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:54.183317 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b51be-9994-450e-bda7-a63e3243fe1a None admin] PATH_INFO: `/v3/users/placement` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:54.191566 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b51be-9994-450e-bda7-a63e3243fe1a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:54.193232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b51be-9994-450e-bda7-a63e3243fe1a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-b74b51be-9994-450e-bda7-a63e3243fe1a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:38:54.207195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:54.208372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:54.209493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:54.209493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:54.209493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:38:54.209493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:38:54.209493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:38:54.209493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:38:54.209493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:38:54.209493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:38:54.209493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 13:38:54.209493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:38:54.209493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 456/930] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:38:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:38:54.214896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c724fc-6fa8-485f-88f6-e45647ba68af None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:54.326796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c724fc-6fa8-485f-88f6-e45647ba68af None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:54.363379 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aacd92-b0bd-442d-adc4-8370fad6fa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:54.364012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aacd92-b0bd-442d-adc4-8370fad6fa1f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:54.364283 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aacd92-b0bd-442d-adc4-8370fad6fa1f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:54.364554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aacd92-b0bd-442d-adc4-8370fad6fa1f None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:54.365290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aacd92-b0bd-442d-adc4-8370fad6fa1f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:54.365627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aacd92-b0bd-442d-adc4-8370fad6fa1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:54.367141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aacd92-b0bd-442d-adc4-8370fad6fa1f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:54.388087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 475/931] 10.222.0.36 () {56 vars in 1143 bytes} [Sat Oct 15 13:38:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:38:54.394373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b51be-9994-450e-bda7-a63e3243fe1a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:54.506371 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b51be-9994-450e-bda7-a63e3243fe1a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:54.517179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:54.543344 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da6b9a-e43e-4e11-8137-648cd70c34cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:54.544111 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da6b9a-e43e-4e11-8137-648cd70c34cc None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:54.544393 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da6b9a-e43e-4e11-8137-648cd70c34cc None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:54.544730 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da6b9a-e43e-4e11-8137-648cd70c34cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:54.615380 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da6b9a-e43e-4e11-8137-648cd70c34cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:54.617000 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da6b9a-e43e-4e11-8137-648cd70c34cc None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:54.731834 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 457/932] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:38:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 13:38:54.733823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aacd92-b0bd-442d-adc4-8370fad6fa1f None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:54.774236 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:54.854162 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aacd92-b0bd-442d-adc4-8370fad6fa1f None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:54.897916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85fda6-1902-4cad-ae4b-a8a97a8be6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:54.898560 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85fda6-1902-4cad-ae4b-a8a97a8be6c7 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:54.898837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85fda6-1902-4cad-ae4b-a8a97a8be6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:54.899114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85fda6-1902-4cad-ae4b-a8a97a8be6c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:54.910412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85fda6-1902-4cad-ae4b-a8a97a8be6c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:54.911886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85fda6-1902-4cad-ae4b-a8a97a8be6c7 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-ce85fda6-1902-4cad-ae4b-a8a97a8be6c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:38:54.927893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:54.929010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:38:54.929926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:38:54.929926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:38:54.929926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:38:54.934333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 476/933] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:38:54.935349 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da6b9a-e43e-4e11-8137-648cd70c34cc None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:54.950990 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:55.053617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da6b9a-e43e-4e11-8137-648cd70c34cc None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:55.095371 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f95f0b-798e-40a4-b254-a6d16ada073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:55.096048 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f95f0b-798e-40a4-b254-a6d16ada073c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:55.096329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f95f0b-798e-40a4-b254-a6d16ada073c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:55.096625 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f95f0b-798e-40a4-b254-a6d16ada073c None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:55.097267 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f95f0b-798e-40a4-b254-a6d16ada073c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:55.097557 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f95f0b-798e-40a4-b254-a6d16ada073c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:55.098922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f95f0b-798e-40a4-b254-a6d16ada073c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:55.116415 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 458/934] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:38:55.119080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85fda6-1902-4cad-ae4b-a8a97a8be6c7 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:55.223608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:55.246869 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85fda6-1902-4cad-ae4b-a8a97a8be6c7 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:55.291164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d49327-495e-4f67-bb47-66cc597dddb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:55.291798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d49327-495e-4f67-bb47-66cc597dddb1 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:55.292102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d49327-495e-4f67-bb47-66cc597dddb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:55.292370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d49327-495e-4f67-bb47-66cc597dddb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:55.299262 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d49327-495e-4f67-bb47-66cc597dddb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:55.300741 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d49327-495e-4f67-bb47-66cc597dddb1 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-d8d49327-495e-4f67-bb47-66cc597dddb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:38:55.312834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:55.313897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:55.314797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:38:55.314797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:38:55.314797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:38:55.314797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:38:55.314797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:38:55.314797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:38:55.314797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:38:55.314797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:38:55.314797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 477/935] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:38:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:38:55.320555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f95f0b-798e-40a4-b254-a6d16ada073c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:55.395661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:55.433433 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f95f0b-798e-40a4-b254-a6d16ada073c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:55.479842 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e9a0f-7b3a-4a83-97ab-2161087404fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:55.480549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e9a0f-7b3a-4a83-97ab-2161087404fb None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:55.480922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e9a0f-7b3a-4a83-97ab-2161087404fb None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:55.481313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e9a0f-7b3a-4a83-97ab-2161087404fb None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:55.482125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e9a0f-7b3a-4a83-97ab-2161087404fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:55.482440 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e9a0f-7b3a-4a83-97ab-2161087404fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:55.484172 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e9a0f-7b3a-4a83-97ab-2161087404fb None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:55.499898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 459/936] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:38:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:38:55.507977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d49327-495e-4f67-bb47-66cc597dddb1 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:55.622020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d49327-495e-4f67-bb47-66cc597dddb1 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:55.667548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc211af-3473-43fe-a13c-ff8c6cc4e642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:55.668246 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc211af-3473-43fe-a13c-ff8c6cc4e642 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:55.668628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc211af-3473-43fe-a13c-ff8c6cc4e642 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:55.668887 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc211af-3473-43fe-a13c-ff8c6cc4e642 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:55.669775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc211af-3473-43fe-a13c-ff8c6cc4e642 None admin] RBAC: Adding query filter params (role.id=8b837ae2ed57479a93acb638b80e2d67, scope.project.id=2a5a64cc8aa04972bb464407cb0312b4, user.id=1c6720518c03413aa8f85e4d6142fe87) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:55.670237 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc211af-3473-43fe-a13c-ff8c6cc4e642 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:55.671891 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc211af-3473-43fe-a13c-ff8c6cc4e642 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:55.679475 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 478/937] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:38:55 2022] GET /identity/v3/role_assignments?user.id=1c6720518c03413aa8f85e4d6142fe87&scope.project.id=2a5a64cc8aa04972bb464407cb0312b4&role.id=8b837ae2ed57479a93acb638b80e2d67 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:38:58.097008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd5e2d-a3b9-42a1-88bb-f92e0b963cc9 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:58.097550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd5e2d-a3b9-42a1-88bb-f92e0b963cc9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:58.097550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd5e2d-a3b9-42a1-88bb-f92e0b963cc9 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:58.098911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 460/938] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:38:58.104450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7edcd-7ce5-459c-8262-4881a75094bd None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:58.104914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7edcd-7ce5-459c-8262-4881a75094bd None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:58.105012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7edcd-7ce5-459c-8262-4881a75094bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:58.124394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-ebe7edcd-7ce5-459c-8262-4881a75094bd None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:38:58.125630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ebe7edcd-7ce5-459c-8262-4881a75094bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:38:58.129000 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:58.200156 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 479/939] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:38:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:38:58.473776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd5e2d-a3b9-42a1-88bb-f92e0b963cc9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:58.598114 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd5e2d-a3b9-42a1-88bb-f92e0b963cc9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:58.629126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:58.636737 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b26d8-7605-409a-8fc7-fefcab196f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:58.637616 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b26d8-7605-409a-8fc7-fefcab196f66 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:58.637901 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b26d8-7605-409a-8fc7-fefcab196f66 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:58.638247 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b26d8-7605-409a-8fc7-fefcab196f66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:58.649800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b26d8-7605-409a-8fc7-fefcab196f66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-263b26d8-7605-409a-8fc7-fefcab196f66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:38:58.655125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:58.656417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:58.657512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:58.657512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:58.657512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:58.657512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:58.657512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:38:58.657512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:38:58.657512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:38:58.657512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:38:58.657512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:38:58.657512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:38:58.657512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:38:58.657512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:38:58.657512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 461/940] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:38:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:38:58.663130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7edcd-7ce5-459c-8262-4881a75094bd None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:58.773801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7edcd-7ce5-459c-8262-4881a75094bd None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:58.810481 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc7d52-d1d2-4809-a446-b31f23dfe3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:58.811084 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc7d52-d1d2-4809-a446-b31f23dfe3f6 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:58.811358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc7d52-d1d2-4809-a446-b31f23dfe3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:58.811623 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc7d52-d1d2-4809-a446-b31f23dfe3f6 None admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:58.812324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc7d52-d1d2-4809-a446-b31f23dfe3f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:58.812622 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc7d52-d1d2-4809-a446-b31f23dfe3f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:58.814029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc7d52-d1d2-4809-a446-b31f23dfe3f6 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:58.823400 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 480/941] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:38:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:38:58.830130 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b26d8-7605-409a-8fc7-fefcab196f66 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:58.954212 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b26d8-7605-409a-8fc7-fefcab196f66 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:58.991595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1233b-a0f7-4c4c-a813-4e8ce7141ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:58.992297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1233b-a0f7-4c4c-a813-4e8ce7141ee1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:58.992571 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1233b-a0f7-4c4c-a813-4e8ce7141ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:58.992838 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1233b-a0f7-4c4c-a813-4e8ce7141ee1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:59.004389 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1233b-a0f7-4c4c-a813-4e8ce7141ee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:59.005996 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1233b-a0f7-4c4c-a813-4e8ce7141ee1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-17a1233b-a0f7-4c4c-a813-4e8ce7141ee1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:38:59.021900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:59.023103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:38:59.024116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:38:59.024116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:38:59.024116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:38:59.024116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 462/942] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:38:59.029679 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc7d52-d1d2-4809-a446-b31f23dfe3f6 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:59.032909 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:59.161241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc7d52-d1d2-4809-a446-b31f23dfe3f6 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:59.208831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6d3e9-5d3c-43bb-937d-5a62f5619380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:59.209701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6d3e9-5d3c-43bb-937d-5a62f5619380 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:59.210103 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6d3e9-5d3c-43bb-937d-5a62f5619380 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:59.210712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6d3e9-5d3c-43bb-937d-5a62f5619380 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:59.211677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6d3e9-5d3c-43bb-937d-5a62f5619380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:59.212151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6d3e9-5d3c-43bb-937d-5a62f5619380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:59.213772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6d3e9-5d3c-43bb-937d-5a62f5619380 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:59.229998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 481/943] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:38:59.236950 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1233b-a0f7-4c4c-a813-4e8ce7141ee1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:59.293672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:59.366564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1233b-a0f7-4c4c-a813-4e8ce7141ee1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:59.408421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8e2b4-5b72-4176-9164-7c115d96244d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:59.409105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8e2b4-5b72-4176-9164-7c115d96244d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:59.409385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8e2b4-5b72-4176-9164-7c115d96244d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:59.409751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8e2b4-5b72-4176-9164-7c115d96244d None admin] PATH_INFO: `/v3/users/placement` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:59.420313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8e2b4-5b72-4176-9164-7c115d96244d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:59.422047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8e2b4-5b72-4176-9164-7c115d96244d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1fb8e2b4-5b72-4176-9164-7c115d96244d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:38:59.437460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:59.438667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:59.439973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:59.439973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:59.439973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:38:59.439973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:38:59.439973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:38:59.439973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:38:59.439973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:38:59.439973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:38:59.439973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 13:38:59.439973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:38:59.439973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 463/944] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:38:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:38:59.445234 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6d3e9-5d3c-43bb-937d-5a62f5619380 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:59.525063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:59.569369 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6d3e9-5d3c-43bb-937d-5a62f5619380 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:59.615724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71b831-5af8-462d-b15c-f1a7d24a8683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:59.616821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71b831-5af8-462d-b15c-f1a7d24a8683 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:59.617343 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71b831-5af8-462d-b15c-f1a7d24a8683 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:59.617904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71b831-5af8-462d-b15c-f1a7d24a8683 None admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:59.619208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71b831-5af8-462d-b15c-f1a7d24a8683 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:38:59.619729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71b831-5af8-462d-b15c-f1a7d24a8683 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:59.622392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71b831-5af8-462d-b15c-f1a7d24a8683 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:59.647569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 482/945] 10.222.0.36 () {56 vars in 1143 bytes} [Sat Oct 15 13:38:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:38:59.655065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8e2b4-5b72-4176-9164-7c115d96244d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:59.775694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8e2b4-5b72-4176-9164-7c115d96244d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:38:59.777370 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:38:59.818849 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e0987-0fcc-43fb-80a9-bf5587d363ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:38:59.819813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e0987-0fcc-43fb-80a9-bf5587d363ea None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:38:59.819813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e0987-0fcc-43fb-80a9-bf5587d363ea None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:38:59.820101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e0987-0fcc-43fb-80a9-bf5587d363ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:38:59.832267 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e0987-0fcc-43fb-80a9-bf5587d363ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:38:59.833991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e0987-0fcc-43fb-80a9-bf5587d363ea None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-e26e0987-0fcc-43fb-80a9-bf5587d363ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:38:59.853560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:38:59.854837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:38:59.856181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:38:59.856181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:38:59.856181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:38:59.856181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 464/946] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:38:59.861189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71b831-5af8-462d-b15c-f1a7d24a8683 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:38:59.980567 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71b831-5af8-462d-b15c-f1a7d24a8683 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:00.004717 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:00.020795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce918a-45a8-4712-badb-35133c363767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:00.021538 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce918a-45a8-4712-badb-35133c363767 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:00.021829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce918a-45a8-4712-badb-35133c363767 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:00.022104 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce918a-45a8-4712-badb-35133c363767 None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:00.022855 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce918a-45a8-4712-badb-35133c363767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:39:00.023162 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce918a-45a8-4712-badb-35133c363767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:00.024592 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce918a-45a8-4712-badb-35133c363767 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:00.045142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 483/947] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:39:00.047705 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e0987-0fcc-43fb-80a9-bf5587d363ea None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:00.161944 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e0987-0fcc-43fb-80a9-bf5587d363ea None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:00.206333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25d128-6334-45e8-a051-1118d6209224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:00.207152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25d128-6334-45e8-a051-1118d6209224 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:00.207478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25d128-6334-45e8-a051-1118d6209224 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:00.207687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25d128-6334-45e8-a051-1118d6209224 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:00.216678 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25d128-6334-45e8-a051-1118d6209224 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:00.218163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25d128-6334-45e8-a051-1118d6209224 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-ee25d128-6334-45e8-a051-1118d6209224 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:39:00.232662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:39:00.233835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:39:00.234984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:39:00.234984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:39:00.234984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:39:00.234984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:39:00.234984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:39:00.234984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:39:00.234984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:39:00.234984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:39:00.234984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 465/948] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:39:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:39:00.240196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce918a-45a8-4712-badb-35133c363767 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:00.359458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce918a-45a8-4712-badb-35133c363767 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:00.404140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c9cb9-3b6f-42c8-bccc-789b7e27743d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:00.405302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9cb9-3b6f-42c8-bccc-789b7e27743d None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:00.405302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9cb9-3b6f-42c8-bccc-789b7e27743d None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:00.405508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c9cb9-3b6f-42c8-bccc-789b7e27743d None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:00.406234 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c9cb9-3b6f-42c8-bccc-789b7e27743d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:39:00.406583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c9cb9-3b6f-42c8-bccc-789b7e27743d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:00.408127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c9cb9-3b6f-42c8-bccc-789b7e27743d None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:00.428310 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 484/949] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:39:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:39:00.430192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25d128-6334-45e8-a051-1118d6209224 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:00.448941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:00.577281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25d128-6334-45e8-a051-1118d6209224 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:00.614875 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61751cd8-a4b6-418e-97bc-63c5809ff56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:00.615562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61751cd8-a4b6-418e-97bc-63c5809ff56d None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:00.615837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61751cd8-a4b6-418e-97bc-63c5809ff56d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:00.616228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61751cd8-a4b6-418e-97bc-63c5809ff56d None admin] PATH_INFO: `/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/1c6720518c03413aa8f85e4d6142fe87/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:00.618164 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61751cd8-a4b6-418e-97bc-63c5809ff56d None admin] RBAC: Authorizing `identity:create_grant(project_id=2a5a64cc8aa04972bb464407cb0312b4, user_id=1c6720518c03413aa8f85e4d6142fe87, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:00.619679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61751cd8-a4b6-418e-97bc-63c5809ff56d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:00.636654 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 466/950] 10.222.0.36 () {58 vars in 1439 bytes} [Sat Oct 15 13:39:00 2022] PUT /identity/v3/projects/2a5a64cc8aa04972bb464407cb0312b4/users/1c6720518c03413aa8f85e4d6142fe87/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:39:02.937134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda08d7-354c-4d6a-a3bb-a9c64aafd642 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:02.937724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda08d7-354c-4d6a-a3bb-a9c64aafd642 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:02.937724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda08d7-354c-4d6a-a3bb-a9c64aafd642 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:02.938991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 485/951] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:39:02.946886 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401cc1d-9be1-449e-be35-3147a6bdc92d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:02.947626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401cc1d-9be1-449e-be35-3147a6bdc92d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:02.948197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401cc1d-9be1-449e-be35-3147a6bdc92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:02.973838 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a401cc1d-9be1-449e-be35-3147a6bdc92d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:39:02.975626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a401cc1d-9be1-449e-be35-3147a6bdc92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:39:03.069036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 467/952] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:39:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 15 13:39:03.372717 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda08d7-354c-4d6a-a3bb-a9c64aafd642 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:03.426728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:03.523303 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda08d7-354c-4d6a-a3bb-a9c64aafd642 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:03.582222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f638a74-4153-43c6-9adc-9d35d7a3101b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:03.582921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f638a74-4153-43c6-9adc-9d35d7a3101b None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:03.583346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f638a74-4153-43c6-9adc-9d35d7a3101b None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:03.583607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f638a74-4153-43c6-9adc-9d35d7a3101b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:03.599313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f638a74-4153-43c6-9adc-9d35d7a3101b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-7f638a74-4153-43c6-9adc-9d35d7a3101b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 15 13:39:03.604945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:39:03.607002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:39:03.608700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:39:03.608700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:39:03.608700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:39:03.608700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:39:03.608700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 15 13:39:03.608700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 15 13:39:03.608700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 15 13:39:03.608700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 15 13:39:03.608700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 15 13:39:03.608700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 15 13:39:03.608700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 15 13:39:03.608700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:39:03.608700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 486/953] 10.222.0.36 () {56 vars in 1096 bytes} [Sat Oct 15 13:39:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:39:03.615867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401cc1d-9be1-449e-be35-3147a6bdc92d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:03.621204 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:03.760649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401cc1d-9be1-449e-be35-3147a6bdc92d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:03.808992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045ac09-cc03-4495-bcc4-7c4dc6733ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:03.809857 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045ac09-cc03-4495-bcc4-7c4dc6733ce7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:03.810177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045ac09-cc03-4495-bcc4-7c4dc6733ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:03.810505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045ac09-cc03-4495-bcc4-7c4dc6733ce7 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:03.811301 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045ac09-cc03-4495-bcc4-7c4dc6733ce7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:39:03.811608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045ac09-cc03-4495-bcc4-7c4dc6733ce7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:03.813660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045ac09-cc03-4495-bcc4-7c4dc6733ce7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:03.826285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 468/954] 10.222.0.36 () {56 vars in 1099 bytes} [Sat Oct 15 13:39:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:39:03.833081 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f638a74-4153-43c6-9adc-9d35d7a3101b None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:03.970327 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:03.975207 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f638a74-4153-43c6-9adc-9d35d7a3101b None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:04.026150 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284b7aa-21e0-4f58-87a9-a7701cecf830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:04.026725 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284b7aa-21e0-4f58-87a9-a7701cecf830 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:04.027026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284b7aa-21e0-4f58-87a9-a7701cecf830 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:04.027390 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284b7aa-21e0-4f58-87a9-a7701cecf830 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:04.044149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9284b7aa-21e0-4f58-87a9-a7701cecf830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:04.046332 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9284b7aa-21e0-4f58-87a9-a7701cecf830 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-9284b7aa-21e0-4f58-87a9-a7701cecf830 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:39:04.066874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:39:04.068368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:39:04.069590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:39:04.069590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:39:04.069590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:39:04.069590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 487/955] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:39:04.075780 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045ac09-cc03-4495-bcc4-7c4dc6733ce7 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:04.160675 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:04.204436 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045ac09-cc03-4495-bcc4-7c4dc6733ce7 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:04.249000 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec99a4-f715-4eb2-857b-6f07a8ea7c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:04.249929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec99a4-f715-4eb2-857b-6f07a8ea7c6a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:04.250198 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec99a4-f715-4eb2-857b-6f07a8ea7c6a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:04.250462 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec99a4-f715-4eb2-857b-6f07a8ea7c6a None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:04.251114 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec99a4-f715-4eb2-857b-6f07a8ea7c6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:39:04.251416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec99a4-f715-4eb2-857b-6f07a8ea7c6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:04.254662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec99a4-f715-4eb2-857b-6f07a8ea7c6a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:04.270142 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 469/956] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:39:04.277905 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284b7aa-21e0-4f58-87a9-a7701cecf830 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:04.405953 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284b7aa-21e0-4f58-87a9-a7701cecf830 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:04.453357 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d77d3-02d1-427d-a5f9-7d964e523b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:04.454104 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d77d3-02d1-427d-a5f9-7d964e523b4e None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:04.454384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d77d3-02d1-427d-a5f9-7d964e523b4e None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:04.454688 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d77d3-02d1-427d-a5f9-7d964e523b4e None admin] PATH_INFO: `/v3/users/placement` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:04.468834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d77d3-02d1-427d-a5f9-7d964e523b4e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:04.471389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d77d3-02d1-427d-a5f9-7d964e523b4e None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-3f5d77d3-02d1-427d-a5f9-7d964e523b4e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 15 13:39:04.487522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:39:04.489630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:39:04.491554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:39:04.491554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:39:04.491554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 15 13:39:04.491554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 15 13:39:04.491554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 15 13:39:04.491554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 15 13:39:04.491554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 15 13:39:04.491554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 15 13:39:04.491554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 15 13:39:04.491554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:39:04.491554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 488/957] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:39:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:39:04.498338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec99a4-f715-4eb2-857b-6f07a8ea7c6a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:04.617710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec99a4-f715-4eb2-857b-6f07a8ea7c6a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:04.645592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:04.655335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96188bc5-d63d-4335-8489-f96ed05615e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:04.656105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96188bc5-d63d-4335-8489-f96ed05615e5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:04.656442 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96188bc5-d63d-4335-8489-f96ed05615e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:04.656771 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96188bc5-d63d-4335-8489-f96ed05615e5 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:04.657601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96188bc5-d63d-4335-8489-f96ed05615e5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:39:04.657965 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96188bc5-d63d-4335-8489-f96ed05615e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:04.659963 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96188bc5-d63d-4335-8489-f96ed05615e5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:04.680666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 470/958] 10.222.0.36 () {56 vars in 1143 bytes} [Sat Oct 15 13:39:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:39:04.690149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d77d3-02d1-427d-a5f9-7d964e523b4e None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:04.708399 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:04.809840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d77d3-02d1-427d-a5f9-7d964e523b4e None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:04.854952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2ac1d-3f32-472b-8df0-fadac4aa37ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:04.855582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2ac1d-3f32-472b-8df0-fadac4aa37ed None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:04.855852 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2ac1d-3f32-472b-8df0-fadac4aa37ed None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:04.856151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2ac1d-3f32-472b-8df0-fadac4aa37ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:04.943258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2ac1d-3f32-472b-8df0-fadac4aa37ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:04.945130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2ac1d-3f32-472b-8df0-fadac4aa37ed None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:04.963489 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:05.073531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 489/959] 10.222.0.36 () {58 vars in 1282 bytes} [Sat Oct 15 13:39:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 15 13:39:05.080667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96188bc5-d63d-4335-8489-f96ed05615e5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:05.216145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96188bc5-d63d-4335-8489-f96ed05615e5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:05.255550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4813e98-3fd9-40df-af5e-44f4d04bef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:05.256307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4813e98-3fd9-40df-af5e-44f4d04bef3a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:05.256652 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4813e98-3fd9-40df-af5e-44f4d04bef3a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:05.256953 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4813e98-3fd9-40df-af5e-44f4d04bef3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:05.285352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4813e98-3fd9-40df-af5e-44f4d04bef3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:05.287864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4813e98-3fd9-40df-af5e-44f4d04bef3a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-a4813e98-3fd9-40df-af5e-44f4d04bef3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 15 13:39:05.308601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:39:05.310687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 15 13:39:05.312249 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 15 13:39:05.312249 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 15 13:39:05.312249 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:39:05.312249 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 471/960] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:39:05.318722 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2ac1d-3f32-472b-8df0-fadac4aa37ed None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:05.456089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2ac1d-3f32-472b-8df0-fadac4aa37ed None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:05.472946 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:05.498240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788b641-b6f9-4949-bad1-8b110838b91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:05.498871 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788b641-b6f9-4949-bad1-8b110838b91f None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:05.499195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788b641-b6f9-4949-bad1-8b110838b91f None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:05.499409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788b641-b6f9-4949-bad1-8b110838b91f None admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:05.500099 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788b641-b6f9-4949-bad1-8b110838b91f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:39:05.500413 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788b641-b6f9-4949-bad1-8b110838b91f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:05.502074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788b641-b6f9-4949-bad1-8b110838b91f None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:05.517338 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 490/961] 10.222.0.36 () {56 vars in 1109 bytes} [Sat Oct 15 13:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:39:05.532539 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4813e98-3fd9-40df-af5e-44f4d04bef3a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:05.576134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:05.672072 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4813e98-3fd9-40df-af5e-44f4d04bef3a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:05.734812 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dd0e4-1701-4dda-b8ef-adcd47f7c2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:05.735744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dd0e4-1701-4dda-b8ef-adcd47f7c2cb None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:05.736050 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dd0e4-1701-4dda-b8ef-adcd47f7c2cb None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:05.736333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dd0e4-1701-4dda-b8ef-adcd47f7c2cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:05.743986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524dd0e4-1701-4dda-b8ef-adcd47f7c2cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:05.745456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524dd0e4-1701-4dda-b8ef-adcd47f7c2cb None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-524dd0e4-1701-4dda-b8ef-adcd47f7c2cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 15 13:39:05.760131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:39:05.762018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:39:05.763815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 15 13:39:05.763815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 15 13:39:05.763815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 15 13:39:05.763815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 15 13:39:05.763815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 15 13:39:05.763815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 15 13:39:05.763815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 15 13:39:05.763815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Oct 15 13:39:05.763815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 472/962] 10.222.0.36 () {56 vars in 1111 bytes} [Sat Oct 15 13:39:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:39:05.768395 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788b641-b6f9-4949-bad1-8b110838b91f None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:05.899286 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788b641-b6f9-4949-bad1-8b110838b91f None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:05.942059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32821c8-8da4-4ddf-8ac9-ddc4d91473ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:05.942879 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32821c8-8da4-4ddf-8ac9-ddc4d91473ba None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:05.943364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32821c8-8da4-4ddf-8ac9-ddc4d91473ba None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:05.943777 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32821c8-8da4-4ddf-8ac9-ddc4d91473ba None admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:05.944687 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32821c8-8da4-4ddf-8ac9-ddc4d91473ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:39:05.945015 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32821c8-8da4-4ddf-8ac9-ddc4d91473ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:05.947661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32821c8-8da4-4ddf-8ac9-ddc4d91473ba None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:05.967417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 491/963] 10.222.0.36 () {56 vars in 1148 bytes} [Sat Oct 15 13:39:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:39:05.976405 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dd0e4-1701-4dda-b8ef-adcd47f7c2cb None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:06.101769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:06.116152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dd0e4-1701-4dda-b8ef-adcd47f7c2cb None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:06.161776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa12000-15e5-49f6-8c2c-f32ad6663732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:06.162788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa12000-15e5-49f6-8c2c-f32ad6663732 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:06.163693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa12000-15e5-49f6-8c2c-f32ad6663732 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:06.164206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa12000-15e5-49f6-8c2c-f32ad6663732 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:06.165530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa12000-15e5-49f6-8c2c-f32ad6663732 None admin] RBAC: Adding query filter params (role.id=57740dc2ebe1429294f20dd51592b82a, scope.project.id=2a5a64cc8aa04972bb464407cb0312b4, user.id=1c6720518c03413aa8f85e4d6142fe87) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:39:06.166084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa12000-15e5-49f6-8c2c-f32ad6663732 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:06.169104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa12000-15e5-49f6-8c2c-f32ad6663732 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:06.182205 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 473/964] 10.222.0.36 () {56 vars in 1374 bytes} [Sat Oct 15 13:39:05 2022] GET /identity/v3/role_assignments?user.id=1c6720518c03413aa8f85e4d6142fe87&scope.project.id=2a5a64cc8aa04972bb464407cb0312b4&role.id=57740dc2ebe1429294f20dd51592b82a => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:39:08.452132 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144df9d5-6a37-43ae-a7e0-5ca9a592d0e0 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:08.452690 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144df9d5-6a37-43ae-a7e0-5ca9a592d0e0 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:08.452780 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144df9d5-6a37-43ae-a7e0-5ca9a592d0e0 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:08.454181 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 492/965] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:39:08.460477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2e2a6-9522-40ac-8016-307fefe8ac4c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:08.461572 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2e2a6-9522-40ac-8016-307fefe8ac4c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:08.461572 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2e2a6-9522-40ac-8016-307fefe8ac4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:08.483348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-83d2e2a6-9522-40ac-8016-307fefe8ac4c None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:39:08.484785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-83d2e2a6-9522-40ac-8016-307fefe8ac4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:39:08.584477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 474/966] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:39:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:39:08.853220 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144df9d5-6a37-43ae-a7e0-5ca9a592d0e0 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:08.889467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:08.979722 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144df9d5-6a37-43ae-a7e0-5ca9a592d0e0 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:09.027771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf6fc1-f42d-4f1a-8f99-f01084897459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:09.028458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf6fc1-f42d-4f1a-8f99-f01084897459 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:09.028735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf6fc1-f42d-4f1a-8f99-f01084897459 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:09.029009 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf6fc1-f42d-4f1a-8f99-f01084897459 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:09.029591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf6fc1-f42d-4f1a-8f99-f01084897459 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:09.030985 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf6fc1-f42d-4f1a-8f99-f01084897459 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-2cdf6fc1-f42d-4f1a-8f99-f01084897459 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:39:09.041374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:39:09.042648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:39:09.043497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 493/967] 10.222.0.36 () {56 vars in 1117 bytes} [Sat Oct 15 13:39:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 15 13:39:09.049968 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2e2a6-9522-40ac-8016-307fefe8ac4c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:09.154596 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:09.169142 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2e2a6-9522-40ac-8016-307fefe8ac4c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:09.208783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c812f-9442-4c44-a732-bb6522be220c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:09.209407 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c812f-9442-4c44-a732-bb6522be220c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:09.209686 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c812f-9442-4c44-a732-bb6522be220c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:09.209951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c812f-9442-4c44-a732-bb6522be220c None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:09.210584 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c812f-9442-4c44-a732-bb6522be220c None admin] RBAC: Adding query filter params (name=placement) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:39:09.210877 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c812f-9442-4c44-a732-bb6522be220c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:09.212396 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c812f-9442-4c44-a732-bb6522be220c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:09.217816 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 475/968] 10.222.0.36 () {56 vars in 1116 bytes} [Sat Oct 15 13:39:09 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:39:09.224153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf6fc1-f42d-4f1a-8f99-f01084897459 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:09.334322 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:09.341514 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf6fc1-f42d-4f1a-8f99-f01084897459 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:09.389387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a8e00-da3e-48a9-99d2-f4a282c53108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:09.390409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a8e00-da3e-48a9-99d2-f4a282c53108 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:09.390898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a8e00-da3e-48a9-99d2-f4a282c53108 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:09.391370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a8e00-da3e-48a9-99d2-f4a282c53108 None admin] PATH_INFO: `/v3/services` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:09.392573 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a8e00-da3e-48a9-99d2-f4a282c53108 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:39:09.394276 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a8e00-da3e-48a9-99d2-f4a282c53108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:09.397315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a8e00-da3e-48a9-99d2-f4a282c53108 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:09.406003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 494/969] 10.222.0.36 () {56 vars in 1116 bytes} [Sat Oct 15 13:39:09 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:39:11.738064 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9724f18-c06a-4f63-9f12-66d6fc185f19 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:11.738628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9724f18-c06a-4f63-9f12-66d6fc185f19 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:11.738628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9724f18-c06a-4f63-9f12-66d6fc185f19 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:11.739996 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 476/970] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:39:11.747113 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1c027-2e91-4134-a2d0-6fd84b38c1d5 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:11.747635 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1c027-2e91-4134-a2d0-6fd84b38c1d5 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:11.747635 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1c027-2e91-4134-a2d0-6fd84b38c1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:11.780649 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-e3d1c027-2e91-4134-a2d0-6fd84b38c1d5 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:39:11.781883 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e3d1c027-2e91-4134-a2d0-6fd84b38c1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:39:11.852926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 495/971] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:39:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:39:12.156299 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9724f18-c06a-4f63-9f12-66d6fc185f19 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:12.274634 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9724f18-c06a-4f63-9f12-66d6fc185f19 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:12.310500 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818deb9-a1d2-4461-a9ea-95360d61b74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:12.311123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818deb9-a1d2-4461-a9ea-95360d61b74f None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:12.311396 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818deb9-a1d2-4461-a9ea-95360d61b74f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:12.311661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818deb9-a1d2-4461-a9ea-95360d61b74f None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:12.312342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818deb9-a1d2-4461-a9ea-95360d61b74f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:12.313726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818deb9-a1d2-4461-a9ea-95360d61b74f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:12.330682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 477/972] 10.222.0.36 () {60 vars in 1141 bytes} [Sat Oct 15 13:39:12 2022] POST /identity/v3/services => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:39:14.651385 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344ad24-18b0-46ec-9aa6-7f8d7f75c3d3 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:14.652033 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344ad24-18b0-46ec-9aa6-7f8d7f75c3d3 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:14.652343 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344ad24-18b0-46ec-9aa6-7f8d7f75c3d3 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:14.654113 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 496/973] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:39:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:39:14.660173 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f69b46-b5a2-42c7-bb9f-439cc9453b83 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:14.660690 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f69b46-b5a2-42c7-bb9f-439cc9453b83 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:14.660779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f69b46-b5a2-42c7-bb9f-439cc9453b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:14.680424 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b0f69b46-b5a2-42c7-bb9f-439cc9453b83 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:39:14.681639 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b0f69b46-b5a2-42c7-bb9f-439cc9453b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:39:14.692839 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:14.762059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 478/974] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:39:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:39:15.063633 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344ad24-18b0-46ec-9aa6-7f8d7f75c3d3 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:15.166628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:15.178824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344ad24-18b0-46ec-9aa6-7f8d7f75c3d3 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:15.220215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9622f-6567-4d3c-a8e0-23a26cdf46ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:15.220851 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9622f-6567-4d3c-a8e0-23a26cdf46ae None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:15.221196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9622f-6567-4d3c-a8e0-23a26cdf46ae None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:15.221466 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9622f-6567-4d3c-a8e0-23a26cdf46ae None admin] PATH_INFO: `/v3/services/placement` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:15.222176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9622f-6567-4d3c-a8e0-23a26cdf46ae None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:15.223556 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9622f-6567-4d3c-a8e0-23a26cdf46ae None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-b5f9622f-6567-4d3c-a8e0-23a26cdf46ae None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:39:15.234003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:39:15.235347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:39:15.236960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 497/975] 10.222.0.36 () {56 vars in 1117 bytes} [Sat Oct 15 13:39:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:39:15.242545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f69b46-b5a2-42c7-bb9f-439cc9453b83 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:15.349993 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f69b46-b5a2-42c7-bb9f-439cc9453b83 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:15.386180 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f181a-678a-40f1-a677-56ca9df2d42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:15.386795 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f181a-678a-40f1-a677-56ca9df2d42a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:15.387070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f181a-678a-40f1-a677-56ca9df2d42a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:15.387337 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f181a-678a-40f1-a677-56ca9df2d42a None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:15.387996 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f181a-678a-40f1-a677-56ca9df2d42a None admin] RBAC: Adding query filter params (name=placement) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:39:15.388297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f181a-678a-40f1-a677-56ca9df2d42a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:15.389771 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f181a-678a-40f1-a677-56ca9df2d42a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:15.395529 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 479/976] 10.222.0.36 () {56 vars in 1116 bytes} [Sat Oct 15 13:39:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:39:15.401901 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9622f-6567-4d3c-a8e0-23a26cdf46ae None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:15.514654 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9622f-6567-4d3c-a8e0-23a26cdf46ae None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:15.554931 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eca479-edff-47d0-a926-9f5ce1ff7838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:15.555562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eca479-edff-47d0-a926-9f5ce1ff7838 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:15.555838 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eca479-edff-47d0-a926-9f5ce1ff7838 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:15.556159 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eca479-edff-47d0-a926-9f5ce1ff7838 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:15.556855 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eca479-edff-47d0-a926-9f5ce1ff7838 None admin] RBAC: Adding query filter params (interface=public, service_id=7fcccd8a29014bceb67609ece5de384c, region_id=RegionOne) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:39:15.557164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eca479-edff-47d0-a926-9f5ce1ff7838 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:15.558538 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eca479-edff-47d0-a926-9f5ce1ff7838 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:15.563763 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 498/977] 10.222.0.36 () {56 vars in 1251 bytes} [Sat Oct 15 13:39:15 2022] GET /identity/v3/endpoints?service_id=7fcccd8a29014bceb67609ece5de384c&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:39:15.570149 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f181a-678a-40f1-a677-56ca9df2d42a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:15.575295 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:15.673278 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f181a-678a-40f1-a677-56ca9df2d42a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:15.708275 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7385e7a-9ef8-4182-b793-3f0dbf542423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:15.708872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7385e7a-9ef8-4182-b793-3f0dbf542423 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:15.709152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7385e7a-9ef8-4182-b793-3f0dbf542423 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:15.709414 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7385e7a-9ef8-4182-b793-3f0dbf542423 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:15.710011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7385e7a-9ef8-4182-b793-3f0dbf542423 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:15.711389 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7385e7a-9ef8-4182-b793-3f0dbf542423 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:15.718555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 480/978] 10.222.0.36 () {56 vars in 1087 bytes} [Sat Oct 15 13:39:15 2022] GET /identity/v3/services => generated 2191 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:39:17.863992 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7abe6c-cc31-48ff-b79a-95bf15197b7d None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:17.864598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7abe6c-cc31-48ff-b79a-95bf15197b7d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:17.864598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7abe6c-cc31-48ff-b79a-95bf15197b7d None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:17.865830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 499/979] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:39:17.871375 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b7750-8579-4294-88f5-6b8b7f0fa273 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:17.871829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b7750-8579-4294-88f5-6b8b7f0fa273 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:17.871975 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b7750-8579-4294-88f5-6b8b7f0fa273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:17.893597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-298b7750-8579-4294-88f5-6b8b7f0fa273 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:39:17.894861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-298b7750-8579-4294-88f5-6b8b7f0fa273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:39:17.960588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:17.966908 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 481/980] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:39:17 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 15 13:39:18.228344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7abe6c-cc31-48ff-b79a-95bf15197b7d None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:18.248997 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:18.343903 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7abe6c-cc31-48ff-b79a-95bf15197b7d None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:18.384651 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44974a64-cd28-4a70-9c7a-4643925075c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:18.385316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44974a64-cd28-4a70-9c7a-4643925075c2 None admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:18.385591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44974a64-cd28-4a70-9c7a-4643925075c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:18.385858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44974a64-cd28-4a70-9c7a-4643925075c2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:18.386450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44974a64-cd28-4a70-9c7a-4643925075c2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:18.387843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44974a64-cd28-4a70-9c7a-4643925075c2 None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application [None req-44974a64-cd28-4a70-9c7a-4643925075c2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 15 13:39:18.398750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 15 13:39:18.400070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.get_or_create( Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application with Lock( Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self._enter() Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application return self.creator() Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application created_value = creator( Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: ERROR keystone.server.flask.application Oct 15 13:39:18.400902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 500/981] 10.222.0.36 () {56 vars in 1117 bytes} [Sat Oct 15 13:39:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 15 13:39:18.406570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b7750-8579-4294-88f5-6b8b7f0fa273 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:18.523921 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b7750-8579-4294-88f5-6b8b7f0fa273 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:18.561652 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113cbb8-9cca-4ada-b956-4f07b4882c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:18.562268 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113cbb8-9cca-4ada-b956-4f07b4882c02 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:18.562536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113cbb8-9cca-4ada-b956-4f07b4882c02 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:18.562801 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113cbb8-9cca-4ada-b956-4f07b4882c02 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:18.563428 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113cbb8-9cca-4ada-b956-4f07b4882c02 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:39:18.563757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113cbb8-9cca-4ada-b956-4f07b4882c02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:18.565345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113cbb8-9cca-4ada-b956-4f07b4882c02 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:18.571362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 482/982] 10.222.0.36 () {56 vars in 1116 bytes} [Sat Oct 15 13:39:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:39:18.577499 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44974a64-cd28-4a70-9c7a-4643925075c2 None admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:39:18.670059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:18.691293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44974a64-cd28-4a70-9c7a-4643925075c2 None admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:39:18.729724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2644e-4279-418f-b6f2-be580dcc44ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:39:18.730341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2644e-4279-418f-b6f2-be580dcc44ae None admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:39:18.730618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2644e-4279-418f-b6f2-be580dcc44ae None admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:39:18.730887 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2644e-4279-418f-b6f2-be580dcc44ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:39:18.731564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2644e-4279-418f-b6f2-be580dcc44ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:39:18.732973 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2644e-4279-418f-b6f2-be580dcc44ae None admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:39:18.757205 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 501/983] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:39:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:39:28.568469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=69 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:39:28.751775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:25.671631 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844259e-eb97-4618-af51-fd0f95908397 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:25.672745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844259e-eb97-4618-af51-fd0f95908397 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:25.673239 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844259e-eb97-4618-af51-fd0f95908397 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:25.679660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 483/984] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:40:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:40:25.681909 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb96d6-0bf9-4cd4-8223-27420d15ad0a None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:25.682389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb96d6-0bf9-4cd4-8223-27420d15ad0a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:25.682470 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb96d6-0bf9-4cd4-8223-27420d15ad0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:25.884660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-9edb96d6-0bf9-4cd4-8223-27420d15ad0a None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:40:25.886087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9edb96d6-0bf9-4cd4-8223-27420d15ad0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:26.133166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 502/985] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:40:26.436236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679fe9c-f878-4d57-9228-28b0396acecf None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:26.436600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679fe9c-f878-4d57-9228-28b0396acecf None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:26.436976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679fe9c-f878-4d57-9228-28b0396acecf None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:26.439507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 484/986] 10.222.0.36 () {54 vars in 964 bytes} [Sat Oct 15 13:40:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:40:26.448157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33871390-504e-43ef-b33e-10e06785be8a None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:26.448440 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33871390-504e-43ef-b33e-10e06785be8a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:26.448721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33871390-504e-43ef-b33e-10e06785be8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:26.569406 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-33871390-504e-43ef-b33e-10e06785be8a None None] MFA Rules not processed for user `90985ec8c43a4d7895942731dc1d002c`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:40:26.571300 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-33871390-504e-43ef-b33e-10e06785be8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:26.610795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:26.726092 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 503/987] 10.222.0.36 () {58 vars in 1061 bytes} [Sat Oct 15 13:40:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:40:26.735055 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679fe9c-f878-4d57-9228-28b0396acecf None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:40:26.737884 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d679fe9c-f878-4d57-9228-28b0396acecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:26.872848 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679fe9c-f878-4d57-9228-28b0396acecf None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:40:26.965657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ed954-6977-4bb2-a7f2-2291875c0e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:40:26.966549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ed954-6977-4bb2-a7f2-2291875c0e58 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:26.967025 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ed954-6977-4bb2-a7f2-2291875c0e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:26.967461 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ed954-6977-4bb2-a7f2-2291875c0e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:26.969422 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0f9ed954-6977-4bb2-a7f2-2291875c0e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:27.049597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:27.115425 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ed954-6977-4bb2-a7f2-2291875c0e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:40:27.117403 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ed954-6977-4bb2-a7f2-2291875c0e58 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:40:27.142474 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 485/988] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:40:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:40:29.756473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0189a4e-5bb9-4267-be1a-5c9c21a8e1f1 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:29.757008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0189a4e-5bb9-4267-be1a-5c9c21a8e1f1 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:29.757008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0189a4e-5bb9-4267-be1a-5c9c21a8e1f1 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:29.758283 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 504/989] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:40:29.767354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0b111-83c3-48d2-9891-4b582320be98 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:29.767970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0b111-83c3-48d2-9891-4b582320be98 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:29.767970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0b111-83c3-48d2-9891-4b582320be98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:29.853050 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-dcc0b111-83c3-48d2-9891-4b582320be98 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:40:29.854947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dcc0b111-83c3-48d2-9891-4b582320be98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:29.965004 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 486/990] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:40:30.264034 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c4af3-647e-4aca-b722-7245f8fd210a None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:30.264296 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c4af3-647e-4aca-b722-7245f8fd210a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:30.264571 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c4af3-647e-4aca-b722-7245f8fd210a None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:30.266005 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 505/991] 10.222.0.36 () {54 vars in 964 bytes} [Sat Oct 15 13:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:40:30.272633 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204790f7-0749-4ccd-a1ce-c62f2dcfc8c8 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:30.272922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204790f7-0749-4ccd-a1ce-c62f2dcfc8c8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:30.273178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204790f7-0749-4ccd-a1ce-c62f2dcfc8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:30.285594 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:30.365578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-204790f7-0749-4ccd-a1ce-c62f2dcfc8c8 None None] MFA Rules not processed for user `90985ec8c43a4d7895942731dc1d002c`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:40:30.366829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-204790f7-0749-4ccd-a1ce-c62f2dcfc8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:30.450821 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 487/992] 10.222.0.36 () {58 vars in 1061 bytes} [Sat Oct 15 13:40:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:40:30.460303 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c4af3-647e-4aca-b722-7245f8fd210a None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:40:30.461865 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6e9c4af3-647e-4aca-b722-7245f8fd210a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:30.583577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c4af3-647e-4aca-b722-7245f8fd210a None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:40:30.664696 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:30.667267 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58fa20-c099-4bc4-b73f-a784e6c0e59a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:40:30.668043 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58fa20-c099-4bc4-b73f-a784e6c0e59a service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:30.668364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58fa20-c099-4bc4-b73f-a784e6c0e59a service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:30.669008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58fa20-c099-4bc4-b73f-a784e6c0e59a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:30.671433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-da58fa20-c099-4bc4-b73f-a784e6c0e59a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:30.794303 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58fa20-c099-4bc4-b73f-a784e6c0e59a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:40:30.795597 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58fa20-c099-4bc4-b73f-a784e6c0e59a service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:40:30.814963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 506/993] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:40:33.534723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3ee45-67ba-48d8-97a4-038e3aaddab7 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:33.535411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3ee45-67ba-48d8-97a4-038e3aaddab7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:33.535411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3ee45-67ba-48d8-97a4-038e3aaddab7 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:33.537020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 488/994] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:40:33.542451 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdef496-9d80-468c-b306-899bf6f783ec None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:33.542888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdef496-9d80-468c-b306-899bf6f783ec None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:33.543005 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdef496-9d80-468c-b306-899bf6f783ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:33.618948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-8bdef496-9d80-468c-b306-899bf6f783ec None None] MFA Rules not processed for user `852f3d41db194fdcbd8c6e771be4b167`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:40:33.620897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8bdef496-9d80-468c-b306-899bf6f783ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:33.679894 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:33.728507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 507/995] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:40:33 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:40:34.017797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3ee45-67ba-48d8-97a4-038e3aaddab7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:40:34.019322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-35d3ee45-67ba-48d8-97a4-038e3aaddab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:34.154472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3ee45-67ba-48d8-97a4-038e3aaddab7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:40:34.157435 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:34.239816 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c42c2c-629d-40c8-89d4-5a9f06904b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:40:34.240471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42c2c-629d-40c8-89d4-5a9f06904b80 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:34.240745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42c2c-629d-40c8-89d4-5a9f06904b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:34.241015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42c2c-629d-40c8-89d4-5a9f06904b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:34.242640 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d7c42c2c-629d-40c8-89d4-5a9f06904b80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:34.352040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c42c2c-629d-40c8-89d4-5a9f06904b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:40:34.353505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c42c2c-629d-40c8-89d4-5a9f06904b80 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:40:34.371839 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 489/996] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:40:34 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:40:37.070192 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b79b0f-cdc0-41f5-9a0f-ed433f927768 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:37.070805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b79b0f-cdc0-41f5-9a0f-ed433f927768 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:37.070805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b79b0f-cdc0-41f5-9a0f-ed433f927768 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:37.072096 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 508/997] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:40:37.080542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0511e9-f550-4626-ba8e-19db97170e58 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:37.081021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0511e9-f550-4626-ba8e-19db97170e58 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:37.081021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0511e9-f550-4626-ba8e-19db97170e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:37.129547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:37.158632 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5c0511e9-f550-4626-ba8e-19db97170e58 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:40:37.159771 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5c0511e9-f550-4626-ba8e-19db97170e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:37.267435 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 490/998] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:40:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:40:37.558695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b79b0f-cdc0-41f5-9a0f-ed433f927768 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:40:37.560323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-52b79b0f-cdc0-41f5-9a0f-ed433f927768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:37.673339 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b79b0f-cdc0-41f5-9a0f-ed433f927768 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:40:37.753753 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fb9ed-fa1d-45d1-96d6-09335cab4361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:40:37.754540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb9ed-fa1d-45d1-96d6-09335cab4361 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:37.754820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb9ed-fa1d-45d1-96d6-09335cab4361 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:37.755065 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb9ed-fa1d-45d1-96d6-09335cab4361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:37.757168 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-880fb9ed-fa1d-45d1-96d6-09335cab4361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:37.765407 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:37.875994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fb9ed-fa1d-45d1-96d6-09335cab4361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:40:37.877441 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fb9ed-fa1d-45d1-96d6-09335cab4361 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:40:37.896599 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 509/999] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:40:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:40:40.728533 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b0e04-f358-465c-841d-8b36da8c0762 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:40.729107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b0e04-f358-465c-841d-8b36da8c0762 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:40.729107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b0e04-f358-465c-841d-8b36da8c0762 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:40.730428 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 491/1000] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:40:40.736352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0483c-4608-4e18-9a19-dba7b3dee2bc None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:40.736866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0483c-4608-4e18-9a19-dba7b3dee2bc None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:40.736866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0483c-4608-4e18-9a19-dba7b3dee2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:40.810924 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-7eb0483c-4608-4e18-9a19-dba7b3dee2bc None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:40:40.812180 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7eb0483c-4608-4e18-9a19-dba7b3dee2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:40.881897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:40.932963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 510/1001] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:40:41.209273 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b0e04-f358-465c-841d-8b36da8c0762 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:40:41.210888 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6c8b0e04-f358-465c-841d-8b36da8c0762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:41.327291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b0e04-f358-465c-841d-8b36da8c0762 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:40:41.341164 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:41.406368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bcd05-6fe8-4ea8-a60a-c713183567e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:40:41.407058 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bcd05-6fe8-4ea8-a60a-c713183567e1 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:41.407235 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bcd05-6fe8-4ea8-a60a-c713183567e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:41.407508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bcd05-6fe8-4ea8-a60a-c713183567e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:41.409153 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-056bcd05-6fe8-4ea8-a60a-c713183567e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:41.525495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bcd05-6fe8-4ea8-a60a-c713183567e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:40:41.527534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bcd05-6fe8-4ea8-a60a-c713183567e1 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:40:41.557969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 492/1002] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:40:43.842934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba9947-fa51-4b14-94df-7acf66c85ed1 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:43.843548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba9947-fa51-4b14-94df-7acf66c85ed1 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:43.843548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba9947-fa51-4b14-94df-7acf66c85ed1 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:43.844790 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 511/1003] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:40:43.850850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4203c0-6f4b-465e-b587-b64e06f7e653 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:43.851353 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4203c0-6f4b-465e-b587-b64e06f7e653 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:43.851353 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4203c0-6f4b-465e-b587-b64e06f7e653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:43.910665 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:43.932099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-dc4203c0-6f4b-465e-b587-b64e06f7e653 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:40:43.933666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dc4203c0-6f4b-465e-b587-b64e06f7e653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:44.052416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 493/1004] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:40:44.339987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba9947-fa51-4b14-94df-7acf66c85ed1 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:40:44.341545 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-5bba9947-fa51-4b14-94df-7acf66c85ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:44.455099 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba9947-fa51-4b14-94df-7acf66c85ed1 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:40:44.543654 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17288467-484f-4fba-91ee-b3ac4fe791bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:40:44.544374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17288467-484f-4fba-91ee-b3ac4fe791bb service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:44.544618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17288467-484f-4fba-91ee-b3ac4fe791bb service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:44.544889 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17288467-484f-4fba-91ee-b3ac4fe791bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:44.546723 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-17288467-484f-4fba-91ee-b3ac4fe791bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:44.568802 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:44.673874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17288467-484f-4fba-91ee-b3ac4fe791bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:40:44.675323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17288467-484f-4fba-91ee-b3ac4fe791bb service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:40:44.696583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 512/1005] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:40:46.954470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25568a2-d37f-47f8-9717-49f582ca85d6 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:46.955025 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25568a2-d37f-47f8-9717-49f582ca85d6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:46.955025 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25568a2-d37f-47f8-9717-49f582ca85d6 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:46.956296 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 494/1006] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:40:46.961939 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c2868-794a-4ca7-953f-0763caf04073 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:46.962412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c2868-794a-4ca7-953f-0763caf04073 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:46.962521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c2868-794a-4ca7-953f-0763caf04073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:47.034930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-211c2868-794a-4ca7-953f-0763caf04073 None None] MFA Rules not processed for user `852f3d41db194fdcbd8c6e771be4b167`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:40:47.036127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-211c2868-794a-4ca7-953f-0763caf04073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:47.105073 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:47.140311 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 513/1007] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:40:46 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:40:47.420615 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25568a2-d37f-47f8-9717-49f582ca85d6 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:40:47.422200 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b25568a2-d37f-47f8-9717-49f582ca85d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:47.548146 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25568a2-d37f-47f8-9717-49f582ca85d6 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:40:47.574850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:47.637012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ad562-0fd4-41b2-9ccd-545ecad1506e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:40:47.637717 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ad562-0fd4-41b2-9ccd-545ecad1506e service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:47.638029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ad562-0fd4-41b2-9ccd-545ecad1506e service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:47.638297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ad562-0fd4-41b2-9ccd-545ecad1506e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:47.639944 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a11ad562-0fd4-41b2-9ccd-545ecad1506e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:47.763650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ad562-0fd4-41b2-9ccd-545ecad1506e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:40:47.765081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ad562-0fd4-41b2-9ccd-545ecad1506e service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:40:47.789251 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 495/1008] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:40:47 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:40:50.411208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0e533-61b4-4766-9364-1b65e44e263c None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:50.411849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0e533-61b4-4766-9364-1b65e44e263c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:50.412046 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0e533-61b4-4766-9364-1b65e44e263c None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:50.413343 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 514/1009] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:40:50.423430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6c95d-ec10-409e-b3c9-e8775442b88c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:50.424054 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6c95d-ec10-409e-b3c9-e8775442b88c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:50.424054 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6c95d-ec10-409e-b3c9-e8775442b88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:50.501890 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:50.506048 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6ef6c95d-ec10-409e-b3c9-e8775442b88c None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:40:50.508216 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6ef6c95d-ec10-409e-b3c9-e8775442b88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:50.616290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 496/1010] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:40:50.900948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0e533-61b4-4766-9364-1b65e44e263c None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:40:50.902594 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6fb0e533-61b4-4766-9364-1b65e44e263c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:51.016721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0e533-61b4-4766-9364-1b65e44e263c None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:40:51.107317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646d5b5-50cf-4745-b01c-92e55db9131d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:40:51.108215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646d5b5-50cf-4745-b01c-92e55db9131d service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:51.108534 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646d5b5-50cf-4745-b01c-92e55db9131d service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:51.108827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646d5b5-50cf-4745-b01c-92e55db9131d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:51.111204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e646d5b5-50cf-4745-b01c-92e55db9131d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:51.143999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:51.244438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646d5b5-50cf-4745-b01c-92e55db9131d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:40:51.245907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646d5b5-50cf-4745-b01c-92e55db9131d service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:40:51.267957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 515/1011] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:40:53.527709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b5e6e-8e9a-4bd7-b8fb-a3a5857c8988 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:53.528310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b5e6e-8e9a-4bd7-b8fb-a3a5857c8988 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:53.528310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b5e6e-8e9a-4bd7-b8fb-a3a5857c8988 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:53.529706 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 497/1012] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:40:53.535722 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00740ec-e7e8-45f4-bd4b-787079644e2f None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:53.536293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00740ec-e7e8-45f4-bd4b-787079644e2f None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:53.536293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00740ec-e7e8-45f4-bd4b-787079644e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:53.611175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-d00740ec-e7e8-45f4-bd4b-787079644e2f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:40:53.612820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-d00740ec-e7e8-45f4-bd4b-787079644e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:53.693596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:53.717208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 516/1013] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:40:53.998091 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b5e6e-8e9a-4bd7-b8fb-a3a5857c8988 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:40:53.999650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5b9b5e6e-8e9a-4bd7-b8fb-a3a5857c8988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:54.109894 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b5e6e-8e9a-4bd7-b8fb-a3a5857c8988 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:40:54.144232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:54.195382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267b9ec-d031-44e6-bcd2-c6c76320ecac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:40:54.196107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267b9ec-d031-44e6-bcd2-c6c76320ecac service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:54.196399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267b9ec-d031-44e6-bcd2-c6c76320ecac service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:54.196644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267b9ec-d031-44e6-bcd2-c6c76320ecac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:54.198266 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c267b9ec-d031-44e6-bcd2-c6c76320ecac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:54.312289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267b9ec-d031-44e6-bcd2-c6c76320ecac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:40:54.313747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267b9ec-d031-44e6-bcd2-c6c76320ecac service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:40:54.333147 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 498/1014] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:40:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:40:57.008906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4363eb1-f8b7-4458-97ec-2d9c9032765b None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:57.009467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4363eb1-f8b7-4458-97ec-2d9c9032765b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:57.009467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4363eb1-f8b7-4458-97ec-2d9c9032765b None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:57.010789 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 517/1015] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:40:57.016323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd67f91-028c-461a-8132-829cd3bd32ff None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:57.016764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd67f91-028c-461a-8132-829cd3bd32ff None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:57.016856 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd67f91-028c-461a-8132-829cd3bd32ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:57.096934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6bd67f91-028c-461a-8132-829cd3bd32ff None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:40:57.098228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6bd67f91-028c-461a-8132-829cd3bd32ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:57.108779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:57.203995 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 499/1016] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:40:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:40:57.481992 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4363eb1-f8b7-4458-97ec-2d9c9032765b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:40:57.484301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-d4363eb1-f8b7-4458-97ec-2d9c9032765b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:57.594548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4363eb1-f8b7-4458-97ec-2d9c9032765b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:40:57.678126 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb22b1-9366-4fe4-b33a-4f0eea5e3079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:40:57.678765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb22b1-9366-4fe4-b33a-4f0eea5e3079 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:40:57.679051 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb22b1-9366-4fe4-b33a-4f0eea5e3079 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:40:57.679320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb22b1-9366-4fe4-b33a-4f0eea5e3079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:40:57.680997 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-5fcb22b1-9366-4fe4-b33a-4f0eea5e3079 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:40:57.724276 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:40:57.812109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb22b1-9366-4fe4-b33a-4f0eea5e3079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:40:57.813537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb22b1-9366-4fe4-b33a-4f0eea5e3079 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:40:57.834529 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 518/1017] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:40:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:41:02.553624 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002e19b-ec5d-4abf-a0ee-92452b592959 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:02.554254 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002e19b-ec5d-4abf-a0ee-92452b592959 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:02.554254 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002e19b-ec5d-4abf-a0ee-92452b592959 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:02.555458 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 500/1018] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:02.562118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceacd3-8a92-467a-9fb4-7365cbaf15db None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:02.562937 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceacd3-8a92-467a-9fb4-7365cbaf15db None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:02.563090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceacd3-8a92-467a-9fb4-7365cbaf15db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:02.640044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-95ceacd3-8a92-467a-9fb4-7365cbaf15db None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:02.641423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-95ceacd3-8a92-467a-9fb4-7365cbaf15db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:02.750802 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:02.764484 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 519/1019] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:03.046822 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002e19b-ec5d-4abf-a0ee-92452b592959 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:41:03.048451 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5002e19b-ec5d-4abf-a0ee-92452b592959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:03.173647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002e19b-ec5d-4abf-a0ee-92452b592959 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:41:03.218500 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:03.263198 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07370f1-5137-4e7b-a542-2cab825a0547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:41:03.263819 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07370f1-5137-4e7b-a542-2cab825a0547 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:03.264117 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07370f1-5137-4e7b-a542-2cab825a0547 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:03.264364 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07370f1-5137-4e7b-a542-2cab825a0547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:03.266044 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a07370f1-5137-4e7b-a542-2cab825a0547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:03.397388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07370f1-5137-4e7b-a542-2cab825a0547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:41:03.398863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07370f1-5137-4e7b-a542-2cab825a0547 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:41:03.419718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 501/1020] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:41:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:41:06.592317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446377db-8d20-41ac-87eb-0842c3c5f200 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:06.593098 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446377db-8d20-41ac-87eb-0842c3c5f200 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:06.593098 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446377db-8d20-41ac-87eb-0842c3c5f200 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:06.594307 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 520/1021] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:41:06.601823 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d701c0c-9c70-4529-b3fe-64774016c944 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:06.602424 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d701c0c-9c70-4529-b3fe-64774016c944 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:06.602743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d701c0c-9c70-4529-b3fe-64774016c944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:06.694152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2d701c0c-9c70-4529-b3fe-64774016c944 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:06.695359 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2d701c0c-9c70-4529-b3fe-64774016c944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:06.715287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:06.806277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 502/1022] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:41:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:07.101606 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446377db-8d20-41ac-87eb-0842c3c5f200 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:41:07.103310 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-446377db-8d20-41ac-87eb-0842c3c5f200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:07.220776 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446377db-8d20-41ac-87eb-0842c3c5f200 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:41:07.306139 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b243f43-48fb-4e22-847a-83af51cf2e23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:41:07.306765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b243f43-48fb-4e22-847a-83af51cf2e23 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:07.307038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b243f43-48fb-4e22-847a-83af51cf2e23 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:07.307368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b243f43-48fb-4e22-847a-83af51cf2e23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:07.309261 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1b243f43-48fb-4e22-847a-83af51cf2e23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:07.365474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:07.443395 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b243f43-48fb-4e22-847a-83af51cf2e23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:41:07.444892 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b243f43-48fb-4e22-847a-83af51cf2e23 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:41:07.467824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 521/1023] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:41:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:41:12.207103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efec44d-8c22-447a-a576-fc3933462f79 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:12.207718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efec44d-8c22-447a-a576-fc3933462f79 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:12.207718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efec44d-8c22-447a-a576-fc3933462f79 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:12.209460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 503/1024] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:41:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:41:12.216375 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d73fc-cf22-4f25-bba0-7f28e2c011d8 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:12.216834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d73fc-cf22-4f25-bba0-7f28e2c011d8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:12.217134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d73fc-cf22-4f25-bba0-7f28e2c011d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:12.308699 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-8c4d73fc-cf22-4f25-bba0-7f28e2c011d8 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:12.309963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8c4d73fc-cf22-4f25-bba0-7f28e2c011d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:12.440347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:12.443955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 522/1025] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:41:12.723475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efec44d-8c22-447a-a576-fc3933462f79 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:41:12.725236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0efec44d-8c22-447a-a576-fc3933462f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:12.853374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efec44d-8c22-447a-a576-fc3933462f79 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:41:12.911541 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:12.939997 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d62731-8aae-4d31-b6c3-a112421db04c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:41:12.940843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d62731-8aae-4d31-b6c3-a112421db04c service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:12.941220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d62731-8aae-4d31-b6c3-a112421db04c service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:12.941571 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d62731-8aae-4d31-b6c3-a112421db04c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:12.943459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d1d62731-8aae-4d31-b6c3-a112421db04c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:13.061264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d62731-8aae-4d31-b6c3-a112421db04c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:41:13.062726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d62731-8aae-4d31-b6c3-a112421db04c service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:41:13.082224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 504/1026] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:41:18.979810 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860a4cd-828c-4746-a7bd-4dc30ca102d3 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:18.980714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860a4cd-828c-4746-a7bd-4dc30ca102d3 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:18.980714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860a4cd-828c-4746-a7bd-4dc30ca102d3 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:18.981926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 523/1027] 10.222.0.36 () {54 vars in 921 bytes} [Sat Oct 15 13:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:18.988461 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ef2cf-be9f-44d9-b4bf-e5a4b53820ea None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:18.989274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ef2cf-be9f-44d9-b4bf-e5a4b53820ea None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:18.989274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ef2cf-be9f-44d9-b4bf-e5a4b53820ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:19.108040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5e8ef2cf-be9f-44d9-b4bf-e5a4b53820ea None None] MFA Rules not processed for user `1c6720518c03413aa8f85e4d6142fe87`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:19.109570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5e8ef2cf-be9f-44d9-b4bf-e5a4b53820ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:19.133984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:19.293169 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 505/1028] 10.222.0.36 () {58 vars in 1018 bytes} [Sat Oct 15 13:41:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:24.014505 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc6ec9-55d0-4645-ae6a-06e47fd919ee None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:24.015088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc6ec9-55d0-4645-ae6a-06e47fd919ee None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:24.015088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc6ec9-55d0-4645-ae6a-06e47fd919ee None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:24.016603 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 524/1029] 10.222.0.36 () {54 vars in 921 bytes} [Sat Oct 15 13:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:24.021903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b8aa-8014-46f3-b3e3-c2d86654086f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:24.022388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b8aa-8014-46f3-b3e3-c2d86654086f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:24.022642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b8aa-8014-46f3-b3e3-c2d86654086f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:24.122455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-1005b8aa-8014-46f3-b3e3-c2d86654086f None None] MFA Rules not processed for user `1c6720518c03413aa8f85e4d6142fe87`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:24.123833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1005b8aa-8014-46f3-b3e3-c2d86654086f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:24.217794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:24.264920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 506/1030] 10.222.0.36 () {58 vars in 1018 bytes} [Sat Oct 15 13:41:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:25.975676 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9aed-dc70-417e-95a6-5267ce805839 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:25.976371 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9aed-dc70-417e-95a6-5267ce805839 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:25.976663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9aed-dc70-417e-95a6-5267ce805839 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:25.979226 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 525/1031] 10.222.0.36 () {54 vars in 921 bytes} [Sat Oct 15 13:41:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:25.987005 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9063d4-08b4-449b-bf56-bbd9f265f684 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:25.987619 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9063d4-08b4-449b-bf56-bbd9f265f684 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:25.987619 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9063d4-08b4-449b-bf56-bbd9f265f684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:26.086478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0f9063d4-08b4-449b-bf56-bbd9f265f684 None None] MFA Rules not processed for user `1c6720518c03413aa8f85e4d6142fe87`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:26.087895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0f9063d4-08b4-449b-bf56-bbd9f265f684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:26.220832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 507/1032] 10.222.0.36 () {58 vars in 1018 bytes} [Sat Oct 15 13:41:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:30.012159 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4300a00-dff8-49fa-85ba-c24aa38be4bd None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:30.012682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4300a00-dff8-49fa-85ba-c24aa38be4bd None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:30.012800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4300a00-dff8-49fa-85ba-c24aa38be4bd None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:30.014407 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 526/1033] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:30.020685 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597c0a0-de0e-4650-b5ab-181bc54beeda None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:30.020685 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597c0a0-de0e-4650-b5ab-181bc54beeda None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:30.022322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597c0a0-de0e-4650-b5ab-181bc54beeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:30.069729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:30.118715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7597c0a0-de0e-4650-b5ab-181bc54beeda None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:30.120225 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7597c0a0-de0e-4650-b5ab-181bc54beeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:30.295102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 508/1034] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:41:30.608966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c684f-fe0a-45da-9667-e20831ea364d None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:30.609240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c684f-fe0a-45da-9667-e20831ea364d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:30.609674 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c684f-fe0a-45da-9667-e20831ea364d None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:30.612315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 527/1035] 10.222.0.36 () {54 vars in 961 bytes} [Sat Oct 15 13:41:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:30.620983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5948dd-289c-40fc-beff-f54ee7aa503b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:30.621558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5948dd-289c-40fc-beff-f54ee7aa503b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:30.622099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5948dd-289c-40fc-beff-f54ee7aa503b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:30.711416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-3c5948dd-289c-40fc-beff-f54ee7aa503b None None] MFA Rules not processed for user `2e13692c1e83413a909d99bb7db2e4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:30.712657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3c5948dd-289c-40fc-beff-f54ee7aa503b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:30.777096 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:30.862277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 509/1036] 10.222.0.36 () {58 vars in 1058 bytes} [Sat Oct 15 13:41:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:30.871079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c684f-fe0a-45da-9667-e20831ea364d None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:41:30.872973 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-027c684f-fe0a-45da-9667-e20831ea364d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:31.093988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c684f-fe0a-45da-9667-e20831ea364d None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:41:31.204256 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:31.254851 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cd95c-d300-4f09-8706-677172d3dd28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:41:31.255982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cd95c-d300-4f09-8706-677172d3dd28 service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:31.256521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cd95c-d300-4f09-8706-677172d3dd28 service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:31.256862 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cd95c-d300-4f09-8706-677172d3dd28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:31.258604 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b44cd95c-d300-4f09-8706-677172d3dd28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:31.444593 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cd95c-d300-4f09-8706-677172d3dd28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:41:31.447817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cd95c-d300-4f09-8706-677172d3dd28 service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:41:31.482517 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 528/1037] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:41:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:41:34.628176 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6d458-f132-4b4f-8d7c-30e2d247a02f None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:34.629193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6d458-f132-4b4f-8d7c-30e2d247a02f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:34.629680 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6d458-f132-4b4f-8d7c-30e2d247a02f None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:34.632081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 510/1038] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:41:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:41:34.640935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95099e-f26e-48ae-8f20-998e44205b2b None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:34.642284 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95099e-f26e-48ae-8f20-998e44205b2b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:34.642975 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95099e-f26e-48ae-8f20-998e44205b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:34.734167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-ff95099e-f26e-48ae-8f20-998e44205b2b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:34.735609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ff95099e-f26e-48ae-8f20-998e44205b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:34.786731 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:34.876844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 529/1039] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:35.164147 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6d458-f132-4b4f-8d7c-30e2d247a02f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:41:35.165658 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-95b6d458-f132-4b4f-8d7c-30e2d247a02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:35.341257 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6d458-f132-4b4f-8d7c-30e2d247a02f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:41:35.352958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:35.487092 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18785d-64ea-48e9-b7ef-344df0bd19b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:41:35.487092 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18785d-64ea-48e9-b7ef-344df0bd19b8 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:35.487092 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18785d-64ea-48e9-b7ef-344df0bd19b8 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:35.487092 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18785d-64ea-48e9-b7ef-344df0bd19b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:35.489302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-df18785d-64ea-48e9-b7ef-344df0bd19b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:35.614665 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18785d-64ea-48e9-b7ef-344df0bd19b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:41:35.616859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18785d-64ea-48e9-b7ef-344df0bd19b8 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:41:35.638031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 511/1040] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:41:38.181891 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522bbf5-8a23-42a2-a49e-72c09a04e1b3 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:38.182518 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522bbf5-8a23-42a2-a49e-72c09a04e1b3 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:38.182518 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522bbf5-8a23-42a2-a49e-72c09a04e1b3 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:38.184021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 530/1041] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:38.189376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4089d-d4b8-42b8-b8f6-39298a5bfe94 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:38.189899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4089d-d4b8-42b8-b8f6-39298a5bfe94 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:38.189899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4089d-d4b8-42b8-b8f6-39298a5bfe94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:38.206069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:38.269923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d1e4089d-d4b8-42b8-b8f6-39298a5bfe94 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:38.271226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d1e4089d-d4b8-42b8-b8f6-39298a5bfe94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:38.431526 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 512/1042] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:38.744698 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b01e27-c46b-4f01-8e3a-d0dfbf7370c8 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:38.745014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b01e27-c46b-4f01-8e3a-d0dfbf7370c8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:38.745478 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b01e27-c46b-4f01-8e3a-d0dfbf7370c8 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:38.747368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 531/1043] 10.222.0.36 () {54 vars in 961 bytes} [Sat Oct 15 13:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:38.754336 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bfbcb-c4b1-42e8-97f3-875ffe3130ba None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:38.754762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bfbcb-c4b1-42e8-97f3-875ffe3130ba None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:38.755130 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bfbcb-c4b1-42e8-97f3-875ffe3130ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:38.868588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-409bfbcb-c4b1-42e8-97f3-875ffe3130ba None None] MFA Rules not processed for user `2e13692c1e83413a909d99bb7db2e4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:38.870584 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-409bfbcb-c4b1-42e8-97f3-875ffe3130ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:38.919363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40751e-678d-4349-bf41-de7131ad3c96 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:38.920037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40751e-678d-4349-bf41-de7131ad3c96 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:38.920613 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40751e-678d-4349-bf41-de7131ad3c96 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:38.923706 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 532/1044] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:41:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:38.929893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a02c9-5ab7-4bae-8cc5-fb0a0c56beae None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:38.930394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a02c9-5ab7-4bae-8cc5-fb0a0c56beae None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:38.930776 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a02c9-5ab7-4bae-8cc5-fb0a0c56beae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:38.950676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:39.019235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-990a02c9-5ab7-4bae-8cc5-fb0a0c56beae None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:39.020528 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-990a02c9-5ab7-4bae-8cc5-fb0a0c56beae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:39.069804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 513/1045] 10.222.0.36 () {58 vars in 1058 bytes} [Sat Oct 15 13:41:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:39.078281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bfbcb-c4b1-42e8-97f3-875ffe3130ba None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:41:39.092406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bfbcb-c4b1-42e8-97f3-875ffe3130ba None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:41:39.178833 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 533/1046] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:39.278179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221acd00-88e5-4f5f-881b-5c1e65cc0011 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:41:39.278875 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221acd00-88e5-4f5f-881b-5c1e65cc0011 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:39.279246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221acd00-88e5-4f5f-881b-5c1e65cc0011 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:39.279603 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221acd00-88e5-4f5f-881b-5c1e65cc0011 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:39.281380 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-221acd00-88e5-4f5f-881b-5c1e65cc0011 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:39.303452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:39.414864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221acd00-88e5-4f5f-881b-5c1e65cc0011 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:41:39.417183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221acd00-88e5-4f5f-881b-5c1e65cc0011 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:41:39.442886 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 514/1047] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:41:39.539458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3755704-4c48-48d5-bf4e-a47284df4520 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:39.539458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3755704-4c48-48d5-bf4e-a47284df4520 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:39.541351 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3755704-4c48-48d5-bf4e-a47284df4520 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:39.543083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 534/1048] 10.222.0.36 () {54 vars in 963 bytes} [Sat Oct 15 13:41:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:39.551919 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae5f0c-215e-4ba9-a8c8-2e2a90b45cfc None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:39.551919 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae5f0c-215e-4ba9-a8c8-2e2a90b45cfc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:39.551919 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae5f0c-215e-4ba9-a8c8-2e2a90b45cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:39.670301 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9cae5f0c-215e-4ba9-a8c8-2e2a90b45cfc None None] MFA Rules not processed for user `5c96e9f796814f1e8079f3e16073c2eb`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:39.671945 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9cae5f0c-215e-4ba9-a8c8-2e2a90b45cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:39.786510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 515/1049] 10.222.0.36 () {58 vars in 1060 bytes} [Sat Oct 15 13:41:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:39.795301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3755704-4c48-48d5-bf4e-a47284df4520 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:41:39.796964 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-a3755704-4c48-48d5-bf4e-a47284df4520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:39.823102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:39.992167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3755704-4c48-48d5-bf4e-a47284df4520 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:41:40.122112 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c248dc-a3a9-4644-b7d2-edc904e824cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:41:40.122815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c248dc-a3a9-4644-b7d2-edc904e824cb service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:40.123154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c248dc-a3a9-4644-b7d2-edc904e824cb service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:40.123486 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c248dc-a3a9-4644-b7d2-edc904e824cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:40.125438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c9c248dc-a3a9-4644-b7d2-edc904e824cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:40.220110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:40.262694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c248dc-a3a9-4644-b7d2-edc904e824cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:41:40.264178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c248dc-a3a9-4644-b7d2-edc904e824cb service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:41:40.285061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 535/1050] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:41:42.011262 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ffe57-b36e-4867-9fcf-b90df56f8381 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:42.012031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ffe57-b36e-4867-9fcf-b90df56f8381 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:42.012919 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ffe57-b36e-4867-9fcf-b90df56f8381 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:42.015070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 516/1051] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:41:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:42.023087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069912b-a4bb-49da-8c8b-3d5635589321 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:42.023934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069912b-a4bb-49da-8c8b-3d5635589321 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:42.023934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069912b-a4bb-49da-8c8b-3d5635589321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:42.109277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-0069912b-a4bb-49da-8c8b-3d5635589321 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:42.110484 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0069912b-a4bb-49da-8c8b-3d5635589321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:42.263193 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 536/1052] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:42.569436 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ffe57-b36e-4867-9fcf-b90df56f8381 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:41:42.572363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b29ffe57-b36e-4867-9fcf-b90df56f8381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:42.586640 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:42.773165 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ffe57-b36e-4867-9fcf-b90df56f8381 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:41:42.876540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:42.932099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb8093-ab44-4deb-ac97-74a16b27bb9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:41:42.932726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb8093-ab44-4deb-ac97-74a16b27bb9b service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:42.932999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb8093-ab44-4deb-ac97-74a16b27bb9b service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:42.933414 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb8093-ab44-4deb-ac97-74a16b27bb9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:42.935060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-86bb8093-ab44-4deb-ac97-74a16b27bb9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:43.065387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb8093-ab44-4deb-ac97-74a16b27bb9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:41:43.068988 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb8093-ab44-4deb-ac97-74a16b27bb9b service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:41:43.097518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 517/1053] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:41:45.883092 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e225-21bb-4aec-98b2-91c13d18a87d None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:45.883755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e225-21bb-4aec-98b2-91c13d18a87d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:45.883961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e225-21bb-4aec-98b2-91c13d18a87d None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:45.885792 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 537/1054] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:41:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:45.891490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419f895-05f0-4e8a-a7a4-b84a850cb2bb None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:45.892003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419f895-05f0-4e8a-a7a4-b84a850cb2bb None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:45.892116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419f895-05f0-4e8a-a7a4-b84a850cb2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:45.967866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6419f895-05f0-4e8a-a7a4-b84a850cb2bb None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:45.969422 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6419f895-05f0-4e8a-a7a4-b84a850cb2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:46.024596 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:46.094502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 518/1055] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:46.403515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039e225-21bb-4aec-98b2-91c13d18a87d None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:41:46.406275 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0039e225-21bb-4aec-98b2-91c13d18a87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:46.459423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:46.631177 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039e225-21bb-4aec-98b2-91c13d18a87d None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:41:46.805282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:46.838975 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a34003a-28c2-4bfb-8672-53b0826ef44b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:41:46.841559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34003a-28c2-4bfb-8672-53b0826ef44b service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:46.842509 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34003a-28c2-4bfb-8672-53b0826ef44b service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:46.844291 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34003a-28c2-4bfb-8672-53b0826ef44b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:46.847698 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7a34003a-28c2-4bfb-8672-53b0826ef44b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:46.980102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34003a-28c2-4bfb-8672-53b0826ef44b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:41:46.981857 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34003a-28c2-4bfb-8672-53b0826ef44b service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:41:47.004972 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 538/1056] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:41:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:41:49.699423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d3fee-4102-4964-a68d-55fe2832b47d None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:49.700145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d3fee-4102-4964-a68d-55fe2832b47d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:49.700357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d3fee-4102-4964-a68d-55fe2832b47d None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:49.702724 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 519/1057] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:41:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:49.710078 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244788e4-97f4-4b05-83fc-89ba08b7e351 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:49.710717 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244788e4-97f4-4b05-83fc-89ba08b7e351 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:49.710946 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244788e4-97f4-4b05-83fc-89ba08b7e351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:49.823052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-244788e4-97f4-4b05-83fc-89ba08b7e351 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:49.824589 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-244788e4-97f4-4b05-83fc-89ba08b7e351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:49.913589 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:49.983776 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 539/1058] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:50.409683 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d3fee-4102-4964-a68d-55fe2832b47d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:41:50.412935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-092d3fee-4102-4964-a68d-55fe2832b47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:50.624161 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d3fee-4102-4964-a68d-55fe2832b47d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:41:50.655718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:50.778963 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da97f99-4212-498b-858d-dae6223a6ce8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:41:50.780058 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da97f99-4212-498b-858d-dae6223a6ce8 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:50.780604 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da97f99-4212-498b-858d-dae6223a6ce8 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:50.781115 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da97f99-4212-498b-858d-dae6223a6ce8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:50.784438 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3da97f99-4212-498b-858d-dae6223a6ce8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:50.929338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da97f99-4212-498b-858d-dae6223a6ce8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:41:50.931686 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da97f99-4212-498b-858d-dae6223a6ce8 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:41:50.957098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 520/1059] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:41:53.635977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce42cc-fb9f-4807-830c-25fbf51b047a None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:53.636885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce42cc-fb9f-4807-830c-25fbf51b047a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:53.637109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce42cc-fb9f-4807-830c-25fbf51b047a None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:53.640062 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 540/1060] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:41:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:53.650160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca107b-1ec6-4f2b-a861-7b99ddf85fe2 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:53.650767 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca107b-1ec6-4f2b-a861-7b99ddf85fe2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:53.650767 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca107b-1ec6-4f2b-a861-7b99ddf85fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:53.718828 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:53.752597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6bca107b-1ec6-4f2b-a861-7b99ddf85fe2 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:53.753891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6bca107b-1ec6-4f2b-a861-7b99ddf85fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:53.882853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 521/1061] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:54.177266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce42cc-fb9f-4807-830c-25fbf51b047a None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:41:54.180680 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-15ce42cc-fb9f-4807-830c-25fbf51b047a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:54.387188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce42cc-fb9f-4807-830c-25fbf51b047a None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:41:54.461542 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:54.538234 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba30e1-a0e7-416c-8624-1af6aa97b065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:41:54.538849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba30e1-a0e7-416c-8624-1af6aa97b065 service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:54.539130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba30e1-a0e7-416c-8624-1af6aa97b065 service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:54.539395 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba30e1-a0e7-416c-8624-1af6aa97b065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:54.541219 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-85ba30e1-a0e7-416c-8624-1af6aa97b065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:54.661540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba30e1-a0e7-416c-8624-1af6aa97b065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:41:54.664729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba30e1-a0e7-416c-8624-1af6aa97b065 service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:41:54.700065 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 541/1062] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:41:56.685143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2370b-fe2c-4678-b586-00a91957d049 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:56.685838 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2370b-fe2c-4678-b586-00a91957d049 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:56.685838 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2370b-fe2c-4678-b586-00a91957d049 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:56.687014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 522/1063] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:41:56.693048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929f74d-1320-4c47-8c80-4fa85ed449dc None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:56.693600 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929f74d-1320-4c47-8c80-4fa85ed449dc None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:56.693600 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929f74d-1320-4c47-8c80-4fa85ed449dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:56.767182 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-5929f74d-1320-4c47-8c80-4fa85ed449dc None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:56.768456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-5929f74d-1320-4c47-8c80-4fa85ed449dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:56.771403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:56.876920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 542/1064] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:56.879891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1f6bf-3a83-4e07-b080-b4f43c9a60b6 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:56.880191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1f6bf-3a83-4e07-b080-b4f43c9a60b6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:56.880457 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1f6bf-3a83-4e07-b080-b4f43c9a60b6 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:56.882753 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 523/1065] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:41:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:56.888010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b81e9-7da7-41ab-b180-7224e3a06b8d None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:56.888279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b81e9-7da7-41ab-b180-7224e3a06b8d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:56.888553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b81e9-7da7-41ab-b180-7224e3a06b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:56.953774 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-ba4b81e9-7da7-41ab-b180-7224e3a06b8d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:56.954858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ba4b81e9-7da7-41ab-b180-7224e3a06b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:57.030242 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:57.054435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 543/1066] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:57.187959 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b550d-35a4-46bc-ab59-ed3f4840e06b None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:57.188330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b550d-35a4-46bc-ab59-ed3f4840e06b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:57.188658 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b550d-35a4-46bc-ab59-ed3f4840e06b None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:57.190103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 524/1067] 10.222.0.36 () {54 vars in 962 bytes} [Sat Oct 15 13:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:57.196447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866497d5-39b8-4219-af4c-ba04b0e259bb None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:57.196716 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866497d5-39b8-4219-af4c-ba04b0e259bb None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:57.196994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866497d5-39b8-4219-af4c-ba04b0e259bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:57.279133 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-866497d5-39b8-4219-af4c-ba04b0e259bb None None] MFA Rules not processed for user `4f5aabcb1414413a8dce7b76ce701582`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:57.280291 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-866497d5-39b8-4219-af4c-ba04b0e259bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:57.345256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b550d-35a4-46bc-ab59-ed3f4840e06b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:41:57.346844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-619b550d-35a4-46bc-ab59-ed3f4840e06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:57.361842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 544/1068] 10.222.0.36 () {58 vars in 1059 bytes} [Sat Oct 15 13:41:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:57.372006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866497d5-39b8-4219-af4c-ba04b0e259bb None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:41:57.380494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866497d5-39b8-4219-af4c-ba04b0e259bb None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:41:57.455271 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:57.456540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:57.459046 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e4cff-bdb5-4f29-9d19-d3f71efaa328 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f5aabcb1414413a8dce7b76ce701582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:41:57.459863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e4cff-bdb5-4f29-9d19-d3f71efaa328 service glance] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:57.460244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e4cff-bdb5-4f29-9d19-d3f71efaa328 service glance] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:57.460412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e4cff-bdb5-4f29-9d19-d3f71efaa328 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:57.462847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7e6e4cff-bdb5-4f29-9d19-d3f71efaa328 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:57.499595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b550d-35a4-46bc-ab59-ed3f4840e06b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:41:57.586647 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e4cff-bdb5-4f29-9d19-d3f71efaa328 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:41:57.588139 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e4cff-bdb5-4f29-9d19-d3f71efaa328 service glance] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:41:57.607159 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 545/1069] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:41:57.626676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96694b5-d702-46b0-af8a-39e535e1e961 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:41:57.627617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96694b5-d702-46b0-af8a-39e535e1e961 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:57.628004 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96694b5-d702-46b0-af8a-39e535e1e961 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:57.628390 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96694b5-d702-46b0-af8a-39e535e1e961 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:57.631068 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d96694b5-d702-46b0-af8a-39e535e1e961 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:57.722019 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:57.740863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96694b5-d702-46b0-af8a-39e535e1e961 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:41:57.742500 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96694b5-d702-46b0-af8a-39e535e1e961 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:41:57.763417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 525/1070] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:41:58.410494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cc7b2-0db6-4cf5-9113-a4ff76b491c0 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:58.411030 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cc7b2-0db6-4cf5-9113-a4ff76b491c0 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:58.411392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cc7b2-0db6-4cf5-9113-a4ff76b491c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:58.490914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-313cc7b2-0db6-4cf5-9113-a4ff76b491c0 None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:58.492179 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-313cc7b2-0db6-4cf5-9113-a4ff76b491c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:58.554982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:58.593625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 546/1071] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:41:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:58.640448 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f6eea-c7d1-4d07-8fce-3ac346beaa5e None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:58.640683 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f6eea-c7d1-4d07-8fce-3ac346beaa5e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:58.640975 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f6eea-c7d1-4d07-8fce-3ac346beaa5e None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:58.642315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 526/1072] 10.222.0.36 () {54 vars in 943 bytes} [Sat Oct 15 13:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:41:58.653716 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9a7a8-3d05-44d1-bb0b-6d194630fa45 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:58.654114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9a7a8-3d05-44d1-bb0b-6d194630fa45 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:58.654605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9a7a8-3d05-44d1-bb0b-6d194630fa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:58.745525 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-74d9a7a8-3d05-44d1-bb0b-6d194630fa45 None None] MFA Rules not processed for user `d011a40951564323bf7a9c5601df1470`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:41:58.746759 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-74d9a7a8-3d05-44d1-bb0b-6d194630fa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:58.868829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 547/1073] 10.222.0.36 () {58 vars in 1040 bytes} [Sat Oct 15 13:41:58 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:41:58.879354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f6eea-c7d1-4d07-8fce-3ac346beaa5e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:41:58.881018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ec0f6eea-c7d1-4d07-8fce-3ac346beaa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:59.038042 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f6eea-c7d1-4d07-8fce-3ac346beaa5e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:41:59.122153 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:41:59.171632 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1f703-e725-488f-8ffd-53d90d9be73e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:41:59.172547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1f703-e725-488f-8ffd-53d90d9be73e service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:41:59.172547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1f703-e725-488f-8ffd-53d90d9be73e service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:41:59.172816 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1f703-e725-488f-8ffd-53d90d9be73e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:41:59.174420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-03a1f703-e725-488f-8ffd-53d90d9be73e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:41:59.266632 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1f703-e725-488f-8ffd-53d90d9be73e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:41:59.268281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1f703-e725-488f-8ffd-53d90d9be73e service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:41:59.277259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 527/1074] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:41:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:42:00.046334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353817fa-a888-48ea-9434-5fec0c78db9e None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:00.046945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353817fa-a888-48ea-9434-5fec0c78db9e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:00.046945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353817fa-a888-48ea-9434-5fec0c78db9e None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:00.048512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 548/1075] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:00.054443 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4da8f0-9d05-4a86-ba3d-a4456032713d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:00.054913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4da8f0-9d05-4a86-ba3d-a4456032713d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:00.055014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4da8f0-9d05-4a86-ba3d-a4456032713d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:00.128550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ff4da8f0-9d05-4a86-ba3d-a4456032713d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:00.129902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ff4da8f0-9d05-4a86-ba3d-a4456032713d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:00.185881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:00.246839 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 528/1076] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:00.537089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353817fa-a888-48ea-9434-5fec0c78db9e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:00.538816 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-353817fa-a888-48ea-9434-5fec0c78db9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:00.570017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:00.702364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353817fa-a888-48ea-9434-5fec0c78db9e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:00.847050 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:00.868947 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa2a3a-1a23-438f-949b-6d8754f18f5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:00.869621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa2a3a-1a23-438f-949b-6d8754f18f5d service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:00.869840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa2a3a-1a23-438f-949b-6d8754f18f5d service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:00.870114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa2a3a-1a23-438f-949b-6d8754f18f5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:00.871748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-82fa2a3a-1a23-438f-949b-6d8754f18f5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:00.987271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa2a3a-1a23-438f-949b-6d8754f18f5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:00.989368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa2a3a-1a23-438f-949b-6d8754f18f5d service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:01.008847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 549/1077] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:03.338310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231692d-f5e7-453b-8731-25e83faa5eba None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:03.338840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231692d-f5e7-453b-8731-25e83faa5eba None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:03.338840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231692d-f5e7-453b-8731-25e83faa5eba None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:03.340233 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 529/1078] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:03.346517 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bea33b-7bc9-4f1d-80f7-6ac9ad4cf1f0 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:03.347032 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bea33b-7bc9-4f1d-80f7-6ac9ad4cf1f0 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:03.347116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bea33b-7bc9-4f1d-80f7-6ac9ad4cf1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:03.431781 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-15bea33b-7bc9-4f1d-80f7-6ac9ad4cf1f0 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:03.433123 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-15bea33b-7bc9-4f1d-80f7-6ac9ad4cf1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:03.500773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:03.548628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 550/1079] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:03.846243 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231692d-f5e7-453b-8731-25e83faa5eba None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:03.847911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e231692d-f5e7-453b-8731-25e83faa5eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:04.044678 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231692d-f5e7-453b-8731-25e83faa5eba None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:04.084530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:04.182990 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3493a0-7936-4043-90a0-07be509b06cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:04.183613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3493a0-7936-4043-90a0-07be509b06cb service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:04.183909 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3493a0-7936-4043-90a0-07be509b06cb service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:04.184180 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3493a0-7936-4043-90a0-07be509b06cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:04.185866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2d3493a0-7936-4043-90a0-07be509b06cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:04.300479 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3493a0-7936-4043-90a0-07be509b06cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:04.302381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3493a0-7936-4043-90a0-07be509b06cb service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:04.323512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 530/1080] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:06.578772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806e762-2661-4830-8c68-c3e78549052c None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:06.579608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806e762-2661-4830-8c68-c3e78549052c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:06.579608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806e762-2661-4830-8c68-c3e78549052c None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:06.580689 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 551/1081] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:06.587230 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148e05e-1ccc-411f-afcf-41862ac6f1fa None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:06.587747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148e05e-1ccc-411f-afcf-41862ac6f1fa None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:06.587976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148e05e-1ccc-411f-afcf-41862ac6f1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:06.638972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:06.666483 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5148e05e-1ccc-411f-afcf-41862ac6f1fa None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:06.667819 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5148e05e-1ccc-411f-afcf-41862ac6f1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:06.775499 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 531/1082] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:07.069797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806e762-2661-4830-8c68-c3e78549052c None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:07.071352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6806e762-2661-4830-8c68-c3e78549052c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:07.246756 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806e762-2661-4830-8c68-c3e78549052c None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:07.306975 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:07.389097 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe999f52-3d31-457c-82b3-8bd81f25df96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:07.390038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe999f52-3d31-457c-82b3-8bd81f25df96 service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:07.390488 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe999f52-3d31-457c-82b3-8bd81f25df96 service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:07.390756 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe999f52-3d31-457c-82b3-8bd81f25df96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:07.392457 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-fe999f52-3d31-457c-82b3-8bd81f25df96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:07.510492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe999f52-3d31-457c-82b3-8bd81f25df96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:07.512278 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe999f52-3d31-457c-82b3-8bd81f25df96 service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:07.540314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 552/1083] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:42:09.830643 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1d7fd-612d-4aab-8dad-0075e97db477 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:09.831181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1d7fd-612d-4aab-8dad-0075e97db477 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:09.831181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1d7fd-612d-4aab-8dad-0075e97db477 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:09.837015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 532/1084] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:42:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:09.838853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fce4a-31ee-4916-83de-2db0c00e9c42 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:09.839648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fce4a-31ee-4916-83de-2db0c00e9c42 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:09.839966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fce4a-31ee-4916-83de-2db0c00e9c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:09.918736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:09.922361 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-9f7fce4a-31ee-4916-83de-2db0c00e9c42 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:09.925178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9f7fce4a-31ee-4916-83de-2db0c00e9c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:10.042996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 553/1085] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:10.343363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b1d7fd-612d-4aab-8dad-0075e97db477 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:10.344938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-29b1d7fd-612d-4aab-8dad-0075e97db477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:10.508561 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:10.572419 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b1d7fd-612d-4aab-8dad-0075e97db477 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:10.723350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326c9a4-d506-4276-b784-c6eb1c97221d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:10.724001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326c9a4-d506-4276-b784-c6eb1c97221d service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:10.724385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326c9a4-d506-4276-b784-c6eb1c97221d service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:10.724660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326c9a4-d506-4276-b784-c6eb1c97221d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:10.726393 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2326c9a4-d506-4276-b784-c6eb1c97221d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:10.815103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:10.832488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326c9a4-d506-4276-b784-c6eb1c97221d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:10.835051 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326c9a4-d506-4276-b784-c6eb1c97221d service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:10.855261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 533/1086] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:20.041431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=43 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:20.853719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=12 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:22.050310 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16ab64-9df1-4568-bc30-7c6e14062dd8 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:22.050913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16ab64-9df1-4568-bc30-7c6e14062dd8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:22.050913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16ab64-9df1-4568-bc30-7c6e14062dd8 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:22.052388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 554/1087] 10.222.0.36 () {54 vars in 918 bytes} [Sat Oct 15 13:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:22.057892 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3ae58-f8db-4677-804a-274d9ed5ea1e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:22.058400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3ae58-f8db-4677-804a-274d9ed5ea1e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:22.058400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3ae58-f8db-4677-804a-274d9ed5ea1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:22.139728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-eef3ae58-f8db-4677-804a-274d9ed5ea1e None None] MFA Rules not processed for user `1c6720518c03413aa8f85e4d6142fe87`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:22.141027 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-eef3ae58-f8db-4677-804a-274d9ed5ea1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:22.266270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 534/1088] 10.222.0.36 () {58 vars in 1015 bytes} [Sat Oct 15 13:42:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:22.281289 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44d5bc-3b0d-4ff8-b1e6-4228c71150b5 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:22.281571 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44d5bc-3b0d-4ff8-b1e6-4228c71150b5 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:22.281846 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44d5bc-3b0d-4ff8-b1e6-4228c71150b5 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:22.283224 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 555/1089] 10.222.0.36 () {54 vars in 918 bytes} [Sat Oct 15 13:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:42:22.289236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a617f-e197-48b0-b62a-dbaa320ab14e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:22.289526 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a617f-e197-48b0-b62a-dbaa320ab14e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:22.289794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a617f-e197-48b0-b62a-dbaa320ab14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:22.372067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:22.376247 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-660a617f-e197-48b0-b62a-dbaa320ab14e None None] MFA Rules not processed for user `2e13692c1e83413a909d99bb7db2e4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:22.378141 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-660a617f-e197-48b0-b62a-dbaa320ab14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:22.508646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 535/1090] 10.222.0.36 () {58 vars in 1015 bytes} [Sat Oct 15 13:42:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:25.145667 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b1c4c-0398-4699-b64e-8716c88f546b None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:25.146250 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b1c4c-0398-4699-b64e-8716c88f546b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:25.146250 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b1c4c-0398-4699-b64e-8716c88f546b None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:25.147594 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 556/1091] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:25.153195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26d8bf-3898-407a-ae68-fb7221b3637d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:25.153668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26d8bf-3898-407a-ae68-fb7221b3637d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:25.153774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26d8bf-3898-407a-ae68-fb7221b3637d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:25.220657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-fd26d8bf-3898-407a-ae68-fb7221b3637d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:25.221881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fd26d8bf-3898-407a-ae68-fb7221b3637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:25.278458 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:25.323949 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 536/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:25.625801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b1c4c-0398-4699-b64e-8716c88f546b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:25.627333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6e7b1c4c-0398-4699-b64e-8716c88f546b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:25.781379 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b1c4c-0398-4699-b64e-8716c88f546b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:25.859579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:25.908472 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368439f6-cdca-4566-b476-391798c78836 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:25.909097 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368439f6-cdca-4566-b476-391798c78836 service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:25.909346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368439f6-cdca-4566-b476-391798c78836 service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:25.909608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368439f6-cdca-4566-b476-391798c78836 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:25.911457 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-368439f6-cdca-4566-b476-391798c78836 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:26.017646 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368439f6-cdca-4566-b476-391798c78836 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:26.019251 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368439f6-cdca-4566-b476-391798c78836 service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:26.040694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 557/1093] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:27.200850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cec4d-d652-4bbd-b173-2d40f0a1c718 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:27.201409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cec4d-d652-4bbd-b173-2d40f0a1c718 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:27.201409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cec4d-d652-4bbd-b173-2d40f0a1c718 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:27.202652 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 537/1094] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:27.209503 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f0de1-2293-4548-90c3-c90f520820fa None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:27.210022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f0de1-2293-4548-90c3-c90f520820fa None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:27.210022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f0de1-2293-4548-90c3-c90f520820fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:27.292419 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-848f0de1-2293-4548-90c3-c90f520820fa None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:27.293640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-848f0de1-2293-4548-90c3-c90f520820fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:27.322821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:27.396835 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 558/1095] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:27.547529 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8cec4d-d652-4bbd-b173-2d40f0a1c718 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:27.549124 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dc8cec4d-d652-4bbd-b173-2d40f0a1c718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:27.710167 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8cec4d-d652-4bbd-b173-2d40f0a1c718 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:27.755828 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:27.830970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839003f-11e0-41ea-b6f1-de0bf2ff28fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:27.831575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839003f-11e0-41ea-b6f1-de0bf2ff28fb service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:27.831866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839003f-11e0-41ea-b6f1-de0bf2ff28fb service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:27.832131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839003f-11e0-41ea-b6f1-de0bf2ff28fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:27.833772 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b839003f-11e0-41ea-b6f1-de0bf2ff28fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:27.933670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839003f-11e0-41ea-b6f1-de0bf2ff28fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:27.935316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839003f-11e0-41ea-b6f1-de0bf2ff28fb service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:27.955066 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 538/1096] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:29.106034 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0d193-48e1-415a-b5c8-43187590784d None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:29.106598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0d193-48e1-415a-b5c8-43187590784d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:29.106598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0d193-48e1-415a-b5c8-43187590784d None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:29.107980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 559/1097] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:42:29.115478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f190e9a-4c6c-4806-b73b-1d78422c3da0 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:29.116065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f190e9a-4c6c-4806-b73b-1d78422c3da0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:29.116065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f190e9a-4c6c-4806-b73b-1d78422c3da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:29.183188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:29.218630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8f190e9a-4c6c-4806-b73b-1d78422c3da0 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:29.219904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8f190e9a-4c6c-4806-b73b-1d78422c3da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:29.341543 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 539/1098] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:42:29.499461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0d193-48e1-415a-b5c8-43187590784d None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:29.500927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ffd0d193-48e1-415a-b5c8-43187590784d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:29.662631 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0d193-48e1-415a-b5c8-43187590784d None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:29.676213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:29.786575 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe80ce2-ca6f-446b-ae72-24d4aca87121 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:29.787094 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe80ce2-ca6f-446b-ae72-24d4aca87121 service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:29.787370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe80ce2-ca6f-446b-ae72-24d4aca87121 service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:29.787644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe80ce2-ca6f-446b-ae72-24d4aca87121 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:29.789297 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6fe80ce2-ca6f-446b-ae72-24d4aca87121 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:29.884750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe80ce2-ca6f-446b-ae72-24d4aca87121 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:29.886385 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe80ce2-ca6f-446b-ae72-24d4aca87121 service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:29.906789 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 560/1099] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:31.043501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb556a4-4d21-4b6f-9d11-9490a0fcef2f None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:31.044143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb556a4-4d21-4b6f-9d11-9490a0fcef2f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:31.044143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb556a4-4d21-4b6f-9d11-9490a0fcef2f None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:31.045412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 540/1100] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:31.052261 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328f262-5012-4509-a45a-e46ddad39cd3 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:31.052790 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328f262-5012-4509-a45a-e46ddad39cd3 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:31.052790 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328f262-5012-4509-a45a-e46ddad39cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:31.073606 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:31.132436 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-6328f262-5012-4509-a45a-e46ddad39cd3 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:31.133728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6328f262-5012-4509-a45a-e46ddad39cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:31.240798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 561/1101] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:31.390830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb556a4-4d21-4b6f-9d11-9490a0fcef2f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:31.392397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8fb556a4-4d21-4b6f-9d11-9490a0fcef2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:31.519546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:31.559035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb556a4-4d21-4b6f-9d11-9490a0fcef2f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:31.680945 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e86d9a-84d9-4123-919b-d5cf8b9ac2ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:31.681521 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e86d9a-84d9-4123-919b-d5cf8b9ac2ef service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:31.681820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e86d9a-84d9-4123-919b-d5cf8b9ac2ef service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:31.682090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e86d9a-84d9-4123-919b-d5cf8b9ac2ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:31.683740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e0e86d9a-84d9-4123-919b-d5cf8b9ac2ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:31.762960 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:31.777638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e86d9a-84d9-4123-919b-d5cf8b9ac2ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:31.779163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e86d9a-84d9-4123-919b-d5cf8b9ac2ef service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:31.798112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 541/1102] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:32.946385 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6840547-7c0d-4228-813a-e665de760c7b None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:32.946955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6840547-7c0d-4228-813a-e665de760c7b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:32.946955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6840547-7c0d-4228-813a-e665de760c7b None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:32.948322 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 562/1103] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:32.955254 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7969c-9405-4c7e-a3ec-806e41914653 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:32.955757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7969c-9405-4c7e-a3ec-806e41914653 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:32.955757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7969c-9405-4c7e-a3ec-806e41914653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:33.036825 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c5e7969c-9405-4c7e-a3ec-806e41914653 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:33.038028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c5e7969c-9405-4c7e-a3ec-806e41914653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:33.139998 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 542/1104] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:33.290666 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6840547-7c0d-4228-813a-e665de760c7b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:33.292264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b6840547-7c0d-4228-813a-e665de760c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:33.381219 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:33.447552 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6840547-7c0d-4228-813a-e665de760c7b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:33.577199 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb49e2d-91a7-415e-bcd5-47b5a96510c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:33.577809 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb49e2d-91a7-415e-bcd5-47b5a96510c9 service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:33.578088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb49e2d-91a7-415e-bcd5-47b5a96510c9 service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:33.578356 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb49e2d-91a7-415e-bcd5-47b5a96510c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:33.579992 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8fb49e2d-91a7-415e-bcd5-47b5a96510c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:33.630296 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:33.673348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb49e2d-91a7-415e-bcd5-47b5a96510c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:33.675017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb49e2d-91a7-415e-bcd5-47b5a96510c9 service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:33.694273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 563/1105] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:34.754381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bf937-a9b3-47d0-b527-5aa42bcfecb9 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:34.754975 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bf937-a9b3-47d0-b527-5aa42bcfecb9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:34.754975 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bf937-a9b3-47d0-b527-5aa42bcfecb9 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:34.756304 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 543/1106] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:34.763557 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65e48e-ea78-47f1-97a4-499cbf5ab83f None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:34.763557 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65e48e-ea78-47f1-97a4-499cbf5ab83f None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:34.769243 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65e48e-ea78-47f1-97a4-499cbf5ab83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:34.845797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-0c65e48e-ea78-47f1-97a4-499cbf5ab83f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:34.847032 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0c65e48e-ea78-47f1-97a4-499cbf5ab83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:34.948145 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 564/1107] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:35.101067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261bf937-a9b3-47d0-b527-5aa42bcfecb9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:35.102592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-261bf937-a9b3-47d0-b527-5aa42bcfecb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:35.163469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:35.271251 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261bf937-a9b3-47d0-b527-5aa42bcfecb9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:35.391482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb5aab-096e-45ce-b80a-dd79bc8f582d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:35.392013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb5aab-096e-45ce-b80a-dd79bc8f582d service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:35.392285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb5aab-096e-45ce-b80a-dd79bc8f582d service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:35.392542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb5aab-096e-45ce-b80a-dd79bc8f582d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:35.394160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e2fb5aab-096e-45ce-b80a-dd79bc8f582d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:35.402478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:35.499229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb5aab-096e-45ce-b80a-dd79bc8f582d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:35.500920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb5aab-096e-45ce-b80a-dd79bc8f582d service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:35.523912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 544/1108] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:36.606899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf4e8f-3888-4b13-9ac3-28eddf0ff435 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:36.607546 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf4e8f-3888-4b13-9ac3-28eddf0ff435 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:36.607546 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf4e8f-3888-4b13-9ac3-28eddf0ff435 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:36.608938 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 565/1109] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:36.616286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c09c27-07ae-478c-a3cf-8b1a9d31d175 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:36.616840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c09c27-07ae-478c-a3cf-8b1a9d31d175 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:36.616840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c09c27-07ae-478c-a3cf-8b1a9d31d175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:36.696400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c5c09c27-07ae-478c-a3cf-8b1a9d31d175 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:36.697638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c5c09c27-07ae-478c-a3cf-8b1a9d31d175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:36.771511 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:36.801493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 545/1110] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:36.953214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf4e8f-3888-4b13-9ac3-28eddf0ff435 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:36.954757 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c5cf4e8f-3888-4b13-9ac3-28eddf0ff435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:36.973168 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:37.106852 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf4e8f-3888-4b13-9ac3-28eddf0ff435 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:37.200280 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:37.227538 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d5783-b02c-4263-8ded-55f0ff2cf2e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:37.228179 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d5783-b02c-4263-8ded-55f0ff2cf2e3 service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:37.228453 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d5783-b02c-4263-8ded-55f0ff2cf2e3 service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:37.228720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d5783-b02c-4263-8ded-55f0ff2cf2e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:37.230315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-212d5783-b02c-4263-8ded-55f0ff2cf2e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:37.322850 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d5783-b02c-4263-8ded-55f0ff2cf2e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:37.324487 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d5783-b02c-4263-8ded-55f0ff2cf2e3 service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:37.342978 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 566/1111] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:38.419856 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f1739-56b7-4138-8e30-e35db9bfd755 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:38.420545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f1739-56b7-4138-8e30-e35db9bfd755 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:38.420545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f1739-56b7-4138-8e30-e35db9bfd755 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:38.421821 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 546/1112] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:38.429583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e9167-1b52-4735-8f9e-9ed7c11a0765 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:38.430244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e9167-1b52-4735-8f9e-9ed7c11a0765 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:38.430244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e9167-1b52-4735-8f9e-9ed7c11a0765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:38.510911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-331e9167-1b52-4735-8f9e-9ed7c11a0765 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:38.512149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-331e9167-1b52-4735-8f9e-9ed7c11a0765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:38.561532 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:38.623093 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 567/1113] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:38.782662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f1739-56b7-4138-8e30-e35db9bfd755 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:38.784148 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a05f1739-56b7-4138-8e30-e35db9bfd755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:38.937458 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f1739-56b7-4138-8e30-e35db9bfd755 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:38.994525 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:39.054234 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1aad58-880e-49eb-a704-e41cfb6ad416 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:39.055389 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1aad58-880e-49eb-a704-e41cfb6ad416 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:39.055389 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1aad58-880e-49eb-a704-e41cfb6ad416 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:39.055389 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1aad58-880e-49eb-a704-e41cfb6ad416 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:39.056872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-aa1aad58-880e-49eb-a704-e41cfb6ad416 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:39.152112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1aad58-880e-49eb-a704-e41cfb6ad416 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:39.153746 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1aad58-880e-49eb-a704-e41cfb6ad416 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:39.172853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 547/1114] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:40.249721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7161a9b-167f-45c1-8996-24e6f94a3e38 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:40.250312 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7161a9b-167f-45c1-8996-24e6f94a3e38 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:40.250312 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7161a9b-167f-45c1-8996-24e6f94a3e38 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:40.251552 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 568/1115] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:40.258535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4fc93-fb63-40cc-aff1-87866890b36f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:40.259021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4fc93-fb63-40cc-aff1-87866890b36f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:40.259135 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4fc93-fb63-40cc-aff1-87866890b36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:40.340701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-36a4fc93-fb63-40cc-aff1-87866890b36f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:40.341926 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-36a4fc93-fb63-40cc-aff1-87866890b36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:40.344067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:40.444716 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 548/1116] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:40.595508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7161a9b-167f-45c1-8996-24e6f94a3e38 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:40.596956 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e7161a9b-167f-45c1-8996-24e6f94a3e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:40.760083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7161a9b-167f-45c1-8996-24e6f94a3e38 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:40.792023 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:40.894449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdab317-1bf1-4270-8f92-23b1577d3598 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:40.895509 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdab317-1bf1-4270-8f92-23b1577d3598 service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:40.895796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdab317-1bf1-4270-8f92-23b1577d3598 service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:40.896174 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdab317-1bf1-4270-8f92-23b1577d3598 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:40.897776 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7fdab317-1bf1-4270-8f92-23b1577d3598 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:40.990409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdab317-1bf1-4270-8f92-23b1577d3598 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:40.992083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdab317-1bf1-4270-8f92-23b1577d3598 service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:41.010348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 569/1117] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:43.102657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020f5fb-aa5b-46a7-beb3-614cc665411a None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:43.103280 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020f5fb-aa5b-46a7-beb3-614cc665411a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:43.103280 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020f5fb-aa5b-46a7-beb3-614cc665411a None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:43.104617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 549/1118] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:43.110117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17004f-dc66-448e-90cd-130c6e303f9f None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:43.110610 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17004f-dc66-448e-90cd-130c6e303f9f None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:43.110715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17004f-dc66-448e-90cd-130c6e303f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:43.155000 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:43.184658 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-0a17004f-dc66-448e-90cd-130c6e303f9f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:43.185900 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0a17004f-dc66-448e-90cd-130c6e303f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:43.287346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 570/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:43.579271 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020f5fb-aa5b-46a7-beb3-614cc665411a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:43.582216 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8020f5fb-aa5b-46a7-beb3-614cc665411a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:43.700163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020f5fb-aa5b-46a7-beb3-614cc665411a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:43.733926 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:43.790243 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edcf6c-b215-45a7-9586-7474fe667cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:43.790861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edcf6c-b215-45a7-9586-7474fe667cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:43.791158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edcf6c-b215-45a7-9586-7474fe667cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:43.791432 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edcf6c-b215-45a7-9586-7474fe667cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:43.793063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-29edcf6c-b215-45a7-9586-7474fe667cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:43.917238 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edcf6c-b215-45a7-9586-7474fe667cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:43.918701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edcf6c-b215-45a7-9586-7474fe667cc1 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:43.937379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 550/1120] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:46.275370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4418b-3c52-45a8-957d-7db7803a45da None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:46.275963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4418b-3c52-45a8-957d-7db7803a45da None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:46.275963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4418b-3c52-45a8-957d-7db7803a45da None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:46.277307 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 571/1121] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:46.282811 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1f752-6e93-480a-95a6-ac6fd1c6590f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:46.283290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1f752-6e93-480a-95a6-ac6fd1c6590f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:46.283399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1f752-6e93-480a-95a6-ac6fd1c6590f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:46.352642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-12f1f752-6e93-480a-95a6-ac6fd1c6590f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:46.353879 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-12f1f752-6e93-480a-95a6-ac6fd1c6590f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:46.367853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:46.457314 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 551/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:46.738986 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4418b-3c52-45a8-957d-7db7803a45da None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:46.740563 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-5cf4418b-3c52-45a8-957d-7db7803a45da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:46.858297 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4418b-3c52-45a8-957d-7db7803a45da None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:46.877702 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:46.951278 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c38d1c-7505-4987-93bc-6f0791350e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:46.951908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c38d1c-7505-4987-93bc-6f0791350e49 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:46.952183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c38d1c-7505-4987-93bc-6f0791350e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:46.952447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c38d1c-7505-4987-93bc-6f0791350e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:46.954222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-31c38d1c-7505-4987-93bc-6f0791350e49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:47.067241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c38d1c-7505-4987-93bc-6f0791350e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:47.068656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c38d1c-7505-4987-93bc-6f0791350e49 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:47.088116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 572/1123] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:49.592035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb2c4b-5015-4a57-92f2-e11225723751 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:49.592609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb2c4b-5015-4a57-92f2-e11225723751 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:49.592609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb2c4b-5015-4a57-92f2-e11225723751 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:49.593882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 552/1124] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:42:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:49.599631 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d34c9-b627-4afe-9dfe-a48a2216ab0a None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:49.600164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d34c9-b627-4afe-9dfe-a48a2216ab0a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:49.600273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d34c9-b627-4afe-9dfe-a48a2216ab0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:49.653464 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:49.683921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-360d34c9-b627-4afe-9dfe-a48a2216ab0a None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:49.685176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-360d34c9-b627-4afe-9dfe-a48a2216ab0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:49.793362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 573/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:50.082801 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb2c4b-5015-4a57-92f2-e11225723751 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:50.084388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-69bb2c4b-5015-4a57-92f2-e11225723751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:50.204332 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb2c4b-5015-4a57-92f2-e11225723751 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:50.247261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:50.285567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e409d11-ce8e-4a05-a03f-f3b509963c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:50.286181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e409d11-ce8e-4a05-a03f-f3b509963c34 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:50.286547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e409d11-ce8e-4a05-a03f-f3b509963c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:50.286725 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e409d11-ce8e-4a05-a03f-f3b509963c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:50.288393 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9e409d11-ce8e-4a05-a03f-f3b509963c34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:50.410540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e409d11-ce8e-4a05-a03f-f3b509963c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:50.412047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e409d11-ce8e-4a05-a03f-f3b509963c34 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:50.431597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 553/1126] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:52.771682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f2fac-1cc7-4e99-9f9e-51d165547d45 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:52.772461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f2fac-1cc7-4e99-9f9e-51d165547d45 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:52.772461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f2fac-1cc7-4e99-9f9e-51d165547d45 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:52.773629 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 574/1127] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:52.779774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf4808-8f5d-4535-973a-740a4a1aec5a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:52.780288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf4808-8f5d-4535-973a-740a4a1aec5a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:52.780382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf4808-8f5d-4535-973a-740a4a1aec5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:52.849642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-17cf4808-8f5d-4535-973a-740a4a1aec5a None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:52.850852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-17cf4808-8f5d-4535-973a-740a4a1aec5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:52.874758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:52.961918 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 554/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:53.242487 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f2fac-1cc7-4e99-9f9e-51d165547d45 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:53.244080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-587f2fac-1cc7-4e99-9f9e-51d165547d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:53.359755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f2fac-1cc7-4e99-9f9e-51d165547d45 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:53.383694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:53.439988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3f32e-8b38-4b5b-ab66-7d4dc4b83699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:53.440764 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3f32e-8b38-4b5b-ab66-7d4dc4b83699 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:53.441158 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3f32e-8b38-4b5b-ab66-7d4dc4b83699 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:53.441410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3f32e-8b38-4b5b-ab66-7d4dc4b83699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:53.443079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-60a3f32e-8b38-4b5b-ab66-7d4dc4b83699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:53.572121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3f32e-8b38-4b5b-ab66-7d4dc4b83699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:53.573568 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3f32e-8b38-4b5b-ab66-7d4dc4b83699 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:53.593161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 575/1129] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:42:55.858865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b1c35-ee82-40bf-b7b0-be74a2637ecd None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:55.859445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b1c35-ee82-40bf-b7b0-be74a2637ecd None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:55.859445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b1c35-ee82-40bf-b7b0-be74a2637ecd None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:55.860843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 555/1130] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:55.867798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e1a80-af26-4cb8-b85a-a3d7169073cc None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:55.868394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e1a80-af26-4cb8-b85a-a3d7169073cc None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:55.868394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e1a80-af26-4cb8-b85a-a3d7169073cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:55.940157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:55.943921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-937e1a80-af26-4cb8-b85a-a3d7169073cc None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:55.946132 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-937e1a80-af26-4cb8-b85a-a3d7169073cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:56.047607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 576/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:56.332516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b1c35-ee82-40bf-b7b0-be74a2637ecd None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:56.334072 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9f7b1c35-ee82-40bf-b7b0-be74a2637ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:56.442798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b1c35-ee82-40bf-b7b0-be74a2637ecd None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:56.495181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:56.520706 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a8968-61c6-4299-a734-65f60db63939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:56.521311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a8968-61c6-4299-a734-65f60db63939 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:56.521583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a8968-61c6-4299-a734-65f60db63939 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:56.521846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a8968-61c6-4299-a734-65f60db63939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:56.523436 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7e4a8968-61c6-4299-a734-65f60db63939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:56.635480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a8968-61c6-4299-a734-65f60db63939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:56.636970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a8968-61c6-4299-a734-65f60db63939 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:56.656219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 556/1132] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:42:58.924887 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b007b64-8b9b-4746-8d64-cb016615e705 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:58.925514 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b007b64-8b9b-4746-8d64-cb016615e705 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:58.925869 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b007b64-8b9b-4746-8d64-cb016615e705 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:58.927270 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 577/1133] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:42:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:42:58.933236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a726161-e820-4414-9882-3e090ab86645 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:58.933735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a726161-e820-4414-9882-3e090ab86645 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:58.933832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a726161-e820-4414-9882-3e090ab86645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:59.005787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-3a726161-e820-4414-9882-3e090ab86645 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:42:59.007041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3a726161-e820-4414-9882-3e090ab86645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:59.038324 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:59.110651 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 557/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:42:59.392324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b007b64-8b9b-4746-8d64-cb016615e705 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:42:59.393987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1b007b64-8b9b-4746-8d64-cb016615e705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:59.510663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b007b64-8b9b-4746-8d64-cb016615e705 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:42:59.542982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:42:59.594493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10547567-b6dd-4596-975e-90cf7918d47d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:42:59.595809 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10547567-b6dd-4596-975e-90cf7918d47d service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:42:59.596123 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10547567-b6dd-4596-975e-90cf7918d47d service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:42:59.596433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10547567-b6dd-4596-975e-90cf7918d47d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:42:59.598069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-10547567-b6dd-4596-975e-90cf7918d47d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:42:59.723706 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10547567-b6dd-4596-975e-90cf7918d47d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:42:59.725151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10547567-b6dd-4596-975e-90cf7918d47d service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:42:59.744928 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 578/1135] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:43:01.979329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf5c52-250d-4e35-80c5-3e7871318539 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:01.979934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf5c52-250d-4e35-80c5-3e7871318539 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:01.979934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf5c52-250d-4e35-80c5-3e7871318539 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:01.981329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 558/1136] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:43:01.987642 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd623aa-3b90-4d37-b861-ca1be0e806dc None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:01.988120 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd623aa-3b90-4d37-b861-ca1be0e806dc None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:01.988243 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd623aa-3b90-4d37-b861-ca1be0e806dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:02.062698 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-bbd623aa-3b90-4d37-b861-ca1be0e806dc None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:43:02.064564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-bbd623aa-3b90-4d37-b861-ca1be0e806dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:02.078339 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:02.172026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 579/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:43:02.456962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf5c52-250d-4e35-80c5-3e7871318539 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:43:02.458519 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-10cf5c52-250d-4e35-80c5-3e7871318539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:02.573489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf5c52-250d-4e35-80c5-3e7871318539 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:43:02.631054 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:02.649510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d89eec-e6c4-4247-8f68-2fac55254b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:43:02.650126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d89eec-e6c4-4247-8f68-2fac55254b98 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:02.650400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d89eec-e6c4-4247-8f68-2fac55254b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:02.650660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d89eec-e6c4-4247-8f68-2fac55254b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:02.652298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a1d89eec-e6c4-4247-8f68-2fac55254b98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:02.773710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d89eec-e6c4-4247-8f68-2fac55254b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:43:02.775087 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d89eec-e6c4-4247-8f68-2fac55254b98 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:43:02.794832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 559/1138] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:43:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:43:04.989405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a8dc9-8749-404c-8a22-48a7fab968ee None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:04.989988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a8dc9-8749-404c-8a22-48a7fab968ee None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:04.989988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a8dc9-8749-404c-8a22-48a7fab968ee None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:04.991290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 580/1139] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:43:04.997254 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a482f-2637-4336-aa0c-f28015f3b112 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:04.997696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a482f-2637-4336-aa0c-f28015f3b112 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:04.997801 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a482f-2637-4336-aa0c-f28015f3b112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:05.068486 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e98a482f-2637-4336-aa0c-f28015f3b112 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:43:05.069710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e98a482f-2637-4336-aa0c-f28015f3b112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:05.111754 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:05.173156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 560/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:43:05.458517 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a8dc9-8749-404c-8a22-48a7fab968ee None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:43:05.460127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-191a8dc9-8749-404c-8a22-48a7fab968ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:05.575946 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a8dc9-8749-404c-8a22-48a7fab968ee None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:43:05.616843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:05.654673 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559182a-daaf-4f96-bee3-a624d95cd3e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:43:05.655277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559182a-daaf-4f96-bee3-a624d95cd3e1 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:05.655548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559182a-daaf-4f96-bee3-a624d95cd3e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:05.655810 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559182a-daaf-4f96-bee3-a624d95cd3e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:05.657434 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b559182a-daaf-4f96-bee3-a624d95cd3e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:05.772757 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559182a-daaf-4f96-bee3-a624d95cd3e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:43:05.774103 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559182a-daaf-4f96-bee3-a624d95cd3e1 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:43:05.792675 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 581/1141] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:43:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:43:07.996523 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e997a1f-88fd-4eb8-9e83-99c45ada90ef None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:07.997163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e997a1f-88fd-4eb8-9e83-99c45ada90ef None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:07.997163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e997a1f-88fd-4eb8-9e83-99c45ada90ef None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:07.998398 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 561/1142] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:43:08.005010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913fbbf-9526-4608-823d-367cc7acf258 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:08.005612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913fbbf-9526-4608-823d-367cc7acf258 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:08.005954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913fbbf-9526-4608-823d-367cc7acf258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:08.077498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-4913fbbf-9526-4608-823d-367cc7acf258 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:43:08.078721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4913fbbf-9526-4608-823d-367cc7acf258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:08.098443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:08.178937 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 582/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:43:08.466182 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e997a1f-88fd-4eb8-9e83-99c45ada90ef None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:43:08.467717 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2e997a1f-88fd-4eb8-9e83-99c45ada90ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:08.581641 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e997a1f-88fd-4eb8-9e83-99c45ada90ef None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:43:08.655322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:08.663439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b72db9-c3d7-470f-b798-e89fb47b79f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:43:08.664073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b72db9-c3d7-470f-b798-e89fb47b79f4 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:08.664379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b72db9-c3d7-470f-b798-e89fb47b79f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:08.664625 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b72db9-c3d7-470f-b798-e89fb47b79f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:08.666243 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d1b72db9-c3d7-470f-b798-e89fb47b79f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:08.781773 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b72db9-c3d7-470f-b798-e89fb47b79f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:43:08.783143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b72db9-c3d7-470f-b798-e89fb47b79f4 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:43:08.802185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 562/1144] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:43:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:43:11.006000 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008b95-695e-40e7-bde7-36785950aa5a None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:11.006586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008b95-695e-40e7-bde7-36785950aa5a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:11.006586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008b95-695e-40e7-bde7-36785950aa5a None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:11.007919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 583/1145] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:43:11.013466 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87184097-b924-4524-ae02-ef0fa8b988e8 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:11.013930 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87184097-b924-4524-ae02-ef0fa8b988e8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:11.014032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87184097-b924-4524-ae02-ef0fa8b988e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:11.082194 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-87184097-b924-4524-ae02-ef0fa8b988e8 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:43:11.083411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-87184097-b924-4524-ae02-ef0fa8b988e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:11.133428 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:11.186338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 563/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:43:11.465630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2008b95-695e-40e7-bde7-36785950aa5a None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:43:11.467206 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-d2008b95-695e-40e7-bde7-36785950aa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:11.593644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2008b95-695e-40e7-bde7-36785950aa5a None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:43:11.656484 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:11.684780 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221cd17-7211-440e-94be-b955c6b70bab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:43:11.685388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221cd17-7211-440e-94be-b955c6b70bab service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:11.685668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221cd17-7211-440e-94be-b955c6b70bab service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:11.685950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221cd17-7211-440e-94be-b955c6b70bab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:11.687579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0221cd17-7211-440e-94be-b955c6b70bab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:11.814105 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221cd17-7211-440e-94be-b955c6b70bab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:43:11.815911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221cd17-7211-440e-94be-b955c6b70bab service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:43:11.834416 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 584/1147] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:43:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:43:21.184788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=22 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:21.832017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=66 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:32.011385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6747ca-7af3-442a-9042-18237d7409ad None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:32.012008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6747ca-7af3-442a-9042-18237d7409ad None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:32.012008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6747ca-7af3-442a-9042-18237d7409ad None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:32.013290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 564/1148] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:43:32.019033 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb1768-0a39-4b4a-a244-5959afd49d2d None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:32.019511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb1768-0a39-4b4a-a244-5959afd49d2d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:32.019614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb1768-0a39-4b4a-a244-5959afd49d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:32.089982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-a2eb1768-0a39-4b4a-a244-5959afd49d2d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:43:32.091237 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-a2eb1768-0a39-4b4a-a244-5959afd49d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:32.198618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 585/1149] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:43:32.492329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51417d-7b45-4189-9a29-0fe889f75094 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:32.492607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51417d-7b45-4189-9a29-0fe889f75094 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:32.492874 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51417d-7b45-4189-9a29-0fe889f75094 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:32.494255 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 565/1150] 10.222.0.36 () {54 vars in 962 bytes} [Sat Oct 15 13:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:43:32.499582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df80cb-57dc-4e14-b372-e61a3d51c3c0 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:32.499863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df80cb-57dc-4e14-b372-e61a3d51c3c0 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:32.500165 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df80cb-57dc-4e14-b372-e61a3d51c3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:32.578778 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-44df80cb-57dc-4e14-b372-e61a3d51c3c0 None None] MFA Rules not processed for user `4f5aabcb1414413a8dce7b76ce701582`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:43:32.580087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-44df80cb-57dc-4e14-b372-e61a3d51c3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:32.602175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:32.661347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 586/1151] 10.222.0.36 () {58 vars in 1059 bytes} [Sat Oct 15 13:43:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:43:32.669467 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae51417d-7b45-4189-9a29-0fe889f75094 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:43:32.671032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ae51417d-7b45-4189-9a29-0fe889f75094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:32.789211 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae51417d-7b45-4189-9a29-0fe889f75094 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:43:32.868049 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ada6f5-f362-4c8e-971c-9f5b5c9d2924 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f5aabcb1414413a8dce7b76ce701582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:43:32.868683 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ada6f5-f362-4c8e-971c-9f5b5c9d2924 service glance] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:32.868961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ada6f5-f362-4c8e-971c-9f5b5c9d2924 service glance] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:32.869225 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ada6f5-f362-4c8e-971c-9f5b5c9d2924 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:32.870871 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-25ada6f5-f362-4c8e-971c-9f5b5c9d2924 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:32.945827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:33.021914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ada6f5-f362-4c8e-971c-9f5b5c9d2924 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:43:33.023402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ada6f5-f362-4c8e-971c-9f5b5c9d2924 service glance] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:43:33.042717 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 566/1152] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:43:35.316762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061555af-d3a0-4722-89f5-446edadb8c3c None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:35.317308 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061555af-d3a0-4722-89f5-446edadb8c3c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:35.317640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061555af-d3a0-4722-89f5-446edadb8c3c None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:35.320160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 587/1153] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:43:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:43:35.326868 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9130b-1cf2-4482-8946-84caa3ba8208 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:35.327427 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9130b-1cf2-4482-8946-84caa3ba8208 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:35.327606 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9130b-1cf2-4482-8946-84caa3ba8208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:35.414096 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0fe9130b-1cf2-4482-8946-84caa3ba8208 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:43:35.415315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0fe9130b-1cf2-4482-8946-84caa3ba8208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:35.543422 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 567/1154] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:43:35.825454 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061555af-d3a0-4722-89f5-446edadb8c3c None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:43:35.827028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-061555af-d3a0-4722-89f5-446edadb8c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:35.968281 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061555af-d3a0-4722-89f5-446edadb8c3c None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:43:36.025861 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:36.082664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9617e-5a59-4505-966a-89875ff428fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:43:36.083279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9617e-5a59-4505-966a-89875ff428fb admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:36.083552 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9617e-5a59-4505-966a-89875ff428fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:36.083815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9617e-5a59-4505-966a-89875ff428fb admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:36.084572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9617e-5a59-4505-966a-89875ff428fb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:43:36.085695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: WARNING py.warnings [None req-fda9617e-5a59-4505-966a-89875ff428fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:43:36.085695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: warnings.warn(msg) Oct 15 13:43:36.085695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: Oct 15 13:43:36.086866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9617e-5a59-4505-966a-89875ff428fb admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:43:36.108433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 588/1155] 10.222.0.36 () {56 vars in 1108 bytes} [Sat Oct 15 13:43:35 2022] GET /identity/v3/projects => generated 2553 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:43:38.308416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f44d4-6c09-4487-92d8-bcea68395853 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:38.308976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f44d4-6c09-4487-92d8-bcea68395853 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:38.308976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f44d4-6c09-4487-92d8-bcea68395853 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:38.310209 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 568/1156] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:43:38.315921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b5a73-d447-43b5-a201-6c089136b371 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:38.316385 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b5a73-d447-43b5-a201-6c089136b371 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:38.316504 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b5a73-d447-43b5-a201-6c089136b371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:38.385134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-734b5a73-d447-43b5-a201-6c089136b371 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:43:38.386378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-734b5a73-d447-43b5-a201-6c089136b371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:38.490783 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 589/1157] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:43:38.781607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f44d4-6c09-4487-92d8-bcea68395853 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:43:38.783234 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e70f44d4-6c09-4487-92d8-bcea68395853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:38.793235 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:38.939739 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f44d4-6c09-4487-92d8-bcea68395853 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:43:39.024795 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:39.057261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04ff3f-460f-4189-acfe-c0fcc99579d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:43:39.058161 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04ff3f-460f-4189-acfe-c0fcc99579d8 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:39.058435 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04ff3f-460f-4189-acfe-c0fcc99579d8 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:39.058686 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04ff3f-460f-4189-acfe-c0fcc99579d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:39.060325 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5b04ff3f-460f-4189-acfe-c0fcc99579d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:39.156981 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04ff3f-460f-4189-acfe-c0fcc99579d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:43:39.158666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04ff3f-460f-4189-acfe-c0fcc99579d8 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:43:39.179127 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 569/1158] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:43:41.580803 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45178a2d-11db-4840-adbc-02c3b81fcffd None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:41.581403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45178a2d-11db-4840-adbc-02c3b81fcffd None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:41.581403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45178a2d-11db-4840-adbc-02c3b81fcffd None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:41.582714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 590/1159] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:43:41.588561 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fac1d-d3fb-4697-a5ea-1d5a8fd9a568 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:41.589022 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fac1d-d3fb-4697-a5ea-1d5a8fd9a568 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:41.589127 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fac1d-d3fb-4697-a5ea-1d5a8fd9a568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:41.661183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b24fac1d-d3fb-4697-a5ea-1d5a8fd9a568 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:43:41.662418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b24fac1d-d3fb-4697-a5ea-1d5a8fd9a568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:41.711361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:41.765435 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 570/1160] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:43:42.054590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45178a2d-11db-4840-adbc-02c3b81fcffd None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:43:42.056185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-45178a2d-11db-4840-adbc-02c3b81fcffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:42.081880 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:42.167830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45178a2d-11db-4840-adbc-02c3b81fcffd None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:43:42.247085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2630c6f-8bc4-4514-b548-842fdf637bc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:43:42.247716 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2630c6f-8bc4-4514-b548-842fdf637bc7 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:42.248165 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2630c6f-8bc4-4514-b548-842fdf637bc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:42.248325 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2630c6f-8bc4-4514-b548-842fdf637bc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:42.250082 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e2630c6f-8bc4-4514-b548-842fdf637bc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:42.335329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:42.372514 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2630c6f-8bc4-4514-b548-842fdf637bc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:43:42.373953 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2630c6f-8bc4-4514-b548-842fdf637bc7 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:43:42.392897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 591/1161] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:43:44.639361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64dc569-069f-42dc-983d-88c4cfaffa8e None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:44.640030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64dc569-069f-42dc-983d-88c4cfaffa8e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:44.640030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64dc569-069f-42dc-983d-88c4cfaffa8e None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:44.641500 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 571/1162] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:43:44.647235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec1559-7bde-433b-b10b-dfca1e31479c None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:44.647710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec1559-7bde-433b-b10b-dfca1e31479c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:44.647831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec1559-7bde-433b-b10b-dfca1e31479c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:44.720384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-53ec1559-7bde-433b-b10b-dfca1e31479c None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:43:44.721637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-53ec1559-7bde-433b-b10b-dfca1e31479c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:44.836984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 592/1163] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:43:45.121027 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64dc569-069f-42dc-983d-88c4cfaffa8e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:43:45.122665 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a64dc569-069f-42dc-983d-88c4cfaffa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:45.230386 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64dc569-069f-42dc-983d-88c4cfaffa8e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:43:45.308349 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:45.309788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbd0ba-3c96-4e57-8377-ccf84706fa71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:43:45.310396 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbd0ba-3c96-4e57-8377-ccf84706fa71 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:45.310668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbd0ba-3c96-4e57-8377-ccf84706fa71 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:45.310933 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbd0ba-3c96-4e57-8377-ccf84706fa71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:45.313090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6acbd0ba-3c96-4e57-8377-ccf84706fa71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:45.439156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbd0ba-3c96-4e57-8377-ccf84706fa71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:43:45.440659 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbd0ba-3c96-4e57-8377-ccf84706fa71 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:43:45.461060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 572/1164] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:43:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:43:47.813033 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0762e-4811-4785-95e8-c6f877fbae70 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:47.813615 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0762e-4811-4785-95e8-c6f877fbae70 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:47.813874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0762e-4811-4785-95e8-c6f877fbae70 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:47.815514 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 593/1165] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:43:47.821317 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4060c55-8963-48a8-902b-6506fe54e3ce None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:47.821793 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4060c55-8963-48a8-902b-6506fe54e3ce None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:47.821894 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4060c55-8963-48a8-902b-6506fe54e3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:47.894661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b4060c55-8963-48a8-902b-6506fe54e3ce None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:43:47.895909 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b4060c55-8963-48a8-902b-6506fe54e3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:47.953679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:48.000295 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 573/1166] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:43:48.296252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0762e-4811-4785-95e8-c6f877fbae70 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:43:48.297806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-27a0762e-4811-4785-95e8-c6f877fbae70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:48.336978 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:48.410621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0762e-4811-4785-95e8-c6f877fbae70 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:43:48.488196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fea06-0036-4286-93db-85855d2aafea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:43:48.488811 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fea06-0036-4286-93db-85855d2aafea service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:48.489088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fea06-0036-4286-93db-85855d2aafea service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:48.489431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fea06-0036-4286-93db-85855d2aafea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:48.490997 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8a6fea06-0036-4286-93db-85855d2aafea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:48.574541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:48.603914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fea06-0036-4286-93db-85855d2aafea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:43:48.605507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fea06-0036-4286-93db-85855d2aafea service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:43:48.626398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 594/1167] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:43:51.023153 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cecd7-d5e0-437e-9870-3540a006e67e None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:51.023725 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cecd7-d5e0-437e-9870-3540a006e67e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:51.023725 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0cecd7-d5e0-437e-9870-3540a006e67e None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:51.025221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 574/1168] 10.222.0.36 () {54 vars in 927 bytes} [Sat Oct 15 13:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:43:51.030574 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdd85f-e23a-4b36-a498-2ed48c92df20 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:51.030984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdd85f-e23a-4b36-a498-2ed48c92df20 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:51.031129 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdd85f-e23a-4b36-a498-2ed48c92df20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:51.100506 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-95fdd85f-e23a-4b36-a498-2ed48c92df20 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:43:51.101746 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-95fdd85f-e23a-4b36-a498-2ed48c92df20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:51.202893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 595/1169] 10.222.0.36 () {58 vars in 1024 bytes} [Sat Oct 15 13:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:43:51.488537 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cecd7-d5e0-437e-9870-3540a006e67e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:43:51.490090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0c0cecd7-d5e0-437e-9870-3540a006e67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:51.598340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0cecd7-d5e0-437e-9870-3540a006e67e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:43:51.679564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe4e53-ab34-4a01-9941-7b8b4e606a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:43:51.680206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe4e53-ab34-4a01-9941-7b8b4e606a7c service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:43:51.680485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe4e53-ab34-4a01-9941-7b8b4e606a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:43:51.680747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe4e53-ab34-4a01-9941-7b8b4e606a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:43:51.682355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7ffe4e53-ab34-4a01-9941-7b8b4e606a7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:43:51.691427 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:43:51.801014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe4e53-ab34-4a01-9941-7b8b4e606a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:43:51.802362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe4e53-ab34-4a01-9941-7b8b4e606a7c service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:43:51.821766 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 575/1170] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:44:01.201275 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=82 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:44:01.819504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=50 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:45:05.509080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fd0bd-a7ef-45cc-b2d9-4dce31cd9e9e None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:05.509753 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fd0bd-a7ef-45cc-b2d9-4dce31cd9e9e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:05.509753 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fd0bd-a7ef-45cc-b2d9-4dce31cd9e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:05.591173 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-ab8fd0bd-a7ef-45cc-b2d9-4dce31cd9e9e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:45:05.592441 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ab8fd0bd-a7ef-45cc-b2d9-4dce31cd9e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:05.699091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 596/1171] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:45:05.708687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9368a43-c707-4a6f-8a2e-0a73da43ccc8 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:05.709267 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9368a43-c707-4a6f-8a2e-0a73da43ccc8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:05.709556 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9368a43-c707-4a6f-8a2e-0a73da43ccc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:05.839636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e9368a43-c707-4a6f-8a2e-0a73da43ccc8 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:45:05.841397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e9368a43-c707-4a6f-8a2e-0a73da43ccc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:05.964144 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 576/1172] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:45:05.969320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fd0bd-a7ef-45cc-b2d9-4dce31cd9e9e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:45:05.970554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ab8fd0bd-a7ef-45cc-b2d9-4dce31cd9e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:05.982685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fd0bd-a7ef-45cc-b2d9-4dce31cd9e9e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:45:06.067895 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:45:06.104111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5db9e-0c6e-4e75-baf1-8824ce98036b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:45:06.104739 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5db9e-0c6e-4e75-baf1-8824ce98036b admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:06.105024 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5db9e-0c6e-4e75-baf1-8824ce98036b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:06.105316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5db9e-0c6e-4e75-baf1-8824ce98036b admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:06.106345 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5db9e-0c6e-4e75-baf1-8824ce98036b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:45:06.106646 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5db9e-0c6e-4e75-baf1-8824ce98036b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:45:06.107663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: WARNING py.warnings [None req-f2f5db9e-0c6e-4e75-baf1-8824ce98036b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:45:06.107663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: warnings.warn(msg) Oct 15 13:45:06.107663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: Oct 15 13:45:06.108737 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5db9e-0c6e-4e75-baf1-8824ce98036b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:45:06.121860 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 597/1173] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:45:06.132153 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9368a43-c707-4a6f-8a2e-0a73da43ccc8 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:45:06.142123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9368a43-c707-4a6f-8a2e-0a73da43ccc8 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:45:06.207078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:45:06.242333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8580e-c1af-4eb3-a1e9-28e2b6d5544d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:45:06.242946 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8580e-c1af-4eb3-a1e9-28e2b6d5544d admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:06.243218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8580e-c1af-4eb3-a1e9-28e2b6d5544d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:06.243480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8580e-c1af-4eb3-a1e9-28e2b6d5544d admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:06.244131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8580e-c1af-4eb3-a1e9-28e2b6d5544d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:45:06.245264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-78c8580e-c1af-4eb3-a1e9-28e2b6d5544d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:45:06.245264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Oct 15 13:45:06.245264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: Oct 15 13:45:06.246444 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8580e-c1af-4eb3-a1e9-28e2b6d5544d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:45:06.304953 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 577/1174] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:45:06 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:45:06.309455 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5db9e-0c6e-4e75-baf1-8824ce98036b admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:45:06.311051 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f2f5db9e-0c6e-4e75-baf1-8824ce98036b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:06.346237 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5db9e-0c6e-4e75-baf1-8824ce98036b admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:45:06.444775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb9883-dcc6-4844-a33c-ce163c11cbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:45:06.445281 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb9883-dcc6-4844-a33c-ce163c11cbd8 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:06.445534 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb9883-dcc6-4844-a33c-ce163c11cbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:06.445798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb9883-dcc6-4844-a33c-ce163c11cbd8 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:06.446539 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb9883-dcc6-4844-a33c-ce163c11cbd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:45:06.447435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: WARNING py.warnings [None req-1eeb9883-dcc6-4844-a33c-ce163c11cbd8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:45:06.447435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: warnings.warn(msg) Oct 15 13:45:06.447435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: Oct 15 13:45:06.448692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb9883-dcc6-4844-a33c-ce163c11cbd8 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:45:06.497619 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 598/1175] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:45:06 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:45:06.503474 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8580e-c1af-4eb3-a1e9-28e2b6d5544d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:45:06.505280 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-78c8580e-c1af-4eb3-a1e9-28e2b6d5544d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:06.571409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8580e-c1af-4eb3-a1e9-28e2b6d5544d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:45:06.675410 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:45:06.689863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817769f6-e48c-4fdd-912f-8a581b85fa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:45:06.690477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817769f6-e48c-4fdd-912f-8a581b85fa00 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:06.690747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817769f6-e48c-4fdd-912f-8a581b85fa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:06.691007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817769f6-e48c-4fdd-912f-8a581b85fa00 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:06.691646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817769f6-e48c-4fdd-912f-8a581b85fa00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:45:06.693310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817769f6-e48c-4fdd-912f-8a581b85fa00 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:45:06.706387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 578/1176] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:45:06.710532 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb9883-dcc6-4844-a33c-ce163c11cbd8 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:45:06.712142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1eeb9883-dcc6-4844-a33c-ce163c11cbd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:06.724177 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb9883-dcc6-4844-a33c-ce163c11cbd8 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:45:06.750576 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:45:06.823513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db7b2b-b2f7-4ae4-b8dd-52c3a6c4adc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:45:06.824152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db7b2b-b2f7-4ae4-b8dd-52c3a6c4adc7 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:06.824427 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db7b2b-b2f7-4ae4-b8dd-52c3a6c4adc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:06.824693 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db7b2b-b2f7-4ae4-b8dd-52c3a6c4adc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:06.825329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db7b2b-b2f7-4ae4-b8dd-52c3a6c4adc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:45:06.826984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db7b2b-b2f7-4ae4-b8dd-52c3a6c4adc7 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:45:06.840166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 599/1177] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:45:06.844948 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817769f6-e48c-4fdd-912f-8a581b85fa00 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:45:06.846486 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-817769f6-e48c-4fdd-912f-8a581b85fa00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:06.882879 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817769f6-e48c-4fdd-912f-8a581b85fa00 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:45:06.982662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f84ec4-a672-434f-a89c-75e8273c40ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:45:06.983191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f84ec4-a672-434f-a89c-75e8273c40ba admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:06.983452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f84ec4-a672-434f-a89c-75e8273c40ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:06.983958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f84ec4-a672-434f-a89c-75e8273c40ba admin admin] PATH_INFO: `/v3/projects/c493a50ac92b4becac06812036fede82/users/63c1a15a49d5472c9a196e84279cee93/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:07.011831 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f84ec4-a672-434f-a89c-75e8273c40ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=c493a50ac92b4becac06812036fede82, user_id=63c1a15a49d5472c9a196e84279cee93, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:45:07.014618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f84ec4-a672-434f-a89c-75e8273c40ba admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:45:07.044014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 579/1178] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:45:06 2022] PUT /identity/v3/projects/c493a50ac92b4becac06812036fede82/users/63c1a15a49d5472c9a196e84279cee93/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:45:07.055819 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8581a93-fe59-4842-b319-fa5a193cd3e4 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:07.056145 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8581a93-fe59-4842-b319-fa5a193cd3e4 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:07.056425 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8581a93-fe59-4842-b319-fa5a193cd3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:07.116902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-b8581a93-fe59-4842-b319-fa5a193cd3e4 None None] MFA Rules not processed for user `63c1a15a49d5472c9a196e84279cee93`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:45:07.118150 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b8581a93-fe59-4842-b319-fa5a193cd3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:07.201721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 600/1179] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:45:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:45:07.216253 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a753b-ebda-4972-b9de-2b9325f56c46 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:07.216563 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a753b-ebda-4972-b9de-2b9325f56c46 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:07.216817 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a753b-ebda-4972-b9de-2b9325f56c46 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:07.218506 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 580/1180] 10.222.0.36 () {54 vars in 963 bytes} [Sat Oct 15 13:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:45:07.224462 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c24bb4-b271-4e08-844d-f23b311f6b12 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:07.224749 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c24bb4-b271-4e08-844d-f23b311f6b12 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:07.225042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c24bb4-b271-4e08-844d-f23b311f6b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:07.257734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:45:07.303845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-59c24bb4-b271-4e08-844d-f23b311f6b12 None None] MFA Rules not processed for user `5c96e9f796814f1e8079f3e16073c2eb`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:45:07.305104 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-59c24bb4-b271-4e08-844d-f23b311f6b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:07.383213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 601/1181] 10.222.0.36 () {58 vars in 1060 bytes} [Sat Oct 15 13:45:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:45:07.391246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a753b-ebda-4972-b9de-2b9325f56c46 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:45:07.392886 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-637a753b-ebda-4972-b9de-2b9325f56c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:07.446950 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:45:07.504559 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a753b-ebda-4972-b9de-2b9325f56c46 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:45:07.582519 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56083100-059f-420f-be65-96d03e7f38c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:45:07.583012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56083100-059f-420f-be65-96d03e7f38c4 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:07.583289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56083100-059f-420f-be65-96d03e7f38c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:07.583555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56083100-059f-420f-be65-96d03e7f38c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:07.585198 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-56083100-059f-420f-be65-96d03e7f38c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:07.691707 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56083100-059f-420f-be65-96d03e7f38c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:45:07.693190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56083100-059f-420f-be65-96d03e7f38c4 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:45:07.696696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:45:07.711770 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 581/1182] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:45:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:45:07.761258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df092e9-5728-4549-abfd-55bac530049d None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:07.761801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df092e9-5728-4549-abfd-55bac530049d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:07.761801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df092e9-5728-4549-abfd-55bac530049d None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:07.763090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 602/1183] 10.222.0.36 () {52 vars in 830 bytes} [Sat Oct 15 13:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:45:07.767577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56083100-059f-420f-be65-96d03e7f38c4 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:45:07.769354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-56083100-059f-420f-be65-96d03e7f38c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:07.888735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56083100-059f-420f-be65-96d03e7f38c4 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:45:07.990251 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:45:07.990883 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:07.991161 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:07.991426 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] PATH_INFO: `/v3/users/63c1a15a49d5472c9a196e84279cee93` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:08.005619 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63c1a15a49d5472c9a196e84279cee93)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:45:08.006805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:45:08.006805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Oct 15 13:45:08.006805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: Oct 15 13:45:08.008472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:45:08.019155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:45:08.077522 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63c1a15a49d5472c9a196e84279cee93'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:45:08.077959 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63c1a15a49d5472c9a196e84279cee93'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:45:08.078487 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63c1a15a49d5472c9a196e84279cee93'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:45:08.104934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63c1a15a49d5472c9a196e84279cee93'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:45:08.117019 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63c1a15a49d5472c9a196e84279cee93'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:45:08.137477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 582/1184] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:45:07 2022] DELETE /identity/v3/users/63c1a15a49d5472c9a196e84279cee93 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:45:08.142560 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efc444-47b6-43fa-89e8-a74ae2a9f5e8 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:08.142772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efc444-47b6-43fa-89e8-a74ae2a9f5e8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:08.143047 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efc444-47b6-43fa-89e8-a74ae2a9f5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:08.202355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-b0efc444-47b6-43fa-89e8-a74ae2a9f5e8 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:45:08.203563 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b0efc444-47b6-43fa-89e8-a74ae2a9f5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:08.281083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:45:08.295367 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 603/1185] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:45:08.310581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:45:08.312141 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:08.419113 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f9eca-cf87-4278-a7bf-faa694ced727 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:45:08.475016 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:45:08.496448 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45991c72-da06-4bcf-a30d-6f51af95a5d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:45:08.497100 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45991c72-da06-4bcf-a30d-6f51af95a5d8 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:08.497376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45991c72-da06-4bcf-a30d-6f51af95a5d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:08.497644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45991c72-da06-4bcf-a30d-6f51af95a5d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:08.499369 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-45991c72-da06-4bcf-a30d-6f51af95a5d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:08.619850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45991c72-da06-4bcf-a30d-6f51af95a5d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:45:08.621358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45991c72-da06-4bcf-a30d-6f51af95a5d8 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:45:08.641483 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 583/1186] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:45:08.984345 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0efc444-47b6-43fa-89e8-a74ae2a9f5e8 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:45:08.985622 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b0efc444-47b6-43fa-89e8-a74ae2a9f5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:45:08.998235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0efc444-47b6-43fa-89e8-a74ae2a9f5e8 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:45:09.107596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:45:09.108286 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:45:09.108566 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:45:09.108837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd admin admin] PATH_INFO: `/v3/projects/c493a50ac92b4becac06812036fede82` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:45:09.120403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c493a50ac92b4becac06812036fede82)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:45:09.121452 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: WARNING py.warnings [None req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:45:09.121452 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: warnings.warn(msg) Oct 15 13:45:09.121452 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: Oct 15 13:45:09.122683 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:45:09.197403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c493a50ac92b4becac06812036fede82', 'request_id': 'req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:45:09.210837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c493a50ac92b4becac06812036fede82', 'request_id': 'req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:45:09.226560 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c493a50ac92b4becac06812036fede82', 'request_id': 'req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:45:09.252149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd admin admin] The token cache is being invalidate because project c493a50ac92b4becac06812036fede82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:45:09.252359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:45:09.252751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e3120ba2-c160-4c73-8a5c-b8dcd1c126bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:45:09.254241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 604/1187] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:45:08 2022] DELETE /identity/v3/projects/c493a50ac92b4becac06812036fede82 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:45:18.638592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=62 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:45:19.249762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:46:35.097439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65435987-098b-4050-a6b8-7e8847a481c0 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:35.098265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65435987-098b-4050-a6b8-7e8847a481c0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:35.098265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65435987-098b-4050-a6b8-7e8847a481c0 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:35.100308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 584/1188] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:46:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:46:35.107039 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd2bf6-45e6-4564-ab35-457f4c4e38e9 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:35.107515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd2bf6-45e6-4564-ab35-457f4c4e38e9 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:35.107618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd2bf6-45e6-4564-ab35-457f4c4e38e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:35.196894 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-14cd2bf6-45e6-4564-ab35-457f4c4e38e9 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:46:35.197999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-14cd2bf6-45e6-4564-ab35-457f4c4e38e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:46:35.316133 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 605/1189] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:46:35.473303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65435987-098b-4050-a6b8-7e8847a481c0 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:46:35.477138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-65435987-098b-4050-a6b8-7e8847a481c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:46:35.612912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65435987-098b-4050-a6b8-7e8847a481c0 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:46:35.619088 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f08f2-cbef-417a-a5a9-ca7e539393ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:46:35.619714 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f08f2-cbef-417a-a5a9-ca7e539393ed service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:35.620008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f08f2-cbef-417a-a5a9-ca7e539393ed service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:35.620276 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f08f2-cbef-417a-a5a9-ca7e539393ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:35.629388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f08f2-cbef-417a-a5a9-ca7e539393ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:46:35.631009 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f08f2-cbef-417a-a5a9-ca7e539393ed service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:46:35.641034 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 585/1190] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:46:36.721968 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a630319-7846-4c1b-97d2-4ceb7a638c0c None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:36.722939 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a630319-7846-4c1b-97d2-4ceb7a638c0c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:36.723252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a630319-7846-4c1b-97d2-4ceb7a638c0c None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:36.726006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 606/1191] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:46:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:46:36.733185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b660d-f53c-4960-9077-5db88235bdac None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:36.733707 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b660d-f53c-4960-9077-5db88235bdac None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:36.733808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b660d-f53c-4960-9077-5db88235bdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:36.754704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0d8b660d-f53c-4960-9077-5db88235bdac None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:46:36.755972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0d8b660d-f53c-4960-9077-5db88235bdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:46:36.765233 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 586/1192] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:46:36.925441 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a630319-7846-4c1b-97d2-4ceb7a638c0c None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:46:36.939823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a630319-7846-4c1b-97d2-4ceb7a638c0c None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:46:36.945967 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97710ae-61d9-4072-9b55-a10d8555512a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:46:36.946588 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97710ae-61d9-4072-9b55-a10d8555512a service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:36.946858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97710ae-61d9-4072-9b55-a10d8555512a service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:36.947125 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97710ae-61d9-4072-9b55-a10d8555512a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:36.955623 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97710ae-61d9-4072-9b55-a10d8555512a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:46:36.957250 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97710ae-61d9-4072-9b55-a10d8555512a service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:46:36.967068 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 607/1193] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:46:37.483818 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0226b-231d-40d7-9211-3df1ab3437af None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:37.484125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0226b-231d-40d7-9211-3df1ab3437af None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:37.484382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0226b-231d-40d7-9211-3df1ab3437af None None] PATH_INFO: `/v3` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:37.485720 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 587/1194] 10.222.0.40 () {54 vars in 925 bytes} [Sat Oct 15 13:46:37 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:46:37.492925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce25fa7-0c01-4feb-838b-2c7da3070cae None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:37.493214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce25fa7-0c01-4feb-838b-2c7da3070cae None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:37.493484 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce25fa7-0c01-4feb-838b-2c7da3070cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:37.561585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-7ce25fa7-0c01-4feb-838b-2c7da3070cae None None] MFA Rules not processed for user `1c6720518c03413aa8f85e4d6142fe87`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:46:37.562824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7ce25fa7-0c01-4feb-838b-2c7da3070cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:46:37.581665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:46:37.648045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 608/1195] 10.222.0.40 () {58 vars in 1015 bytes} [Sat Oct 15 13:46:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:46:41.245826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c14ca-ece5-4e7c-9109-e8bdb716d7f7 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:41.246304 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c14ca-ece5-4e7c-9109-e8bdb716d7f7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:41.246412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c14ca-ece5-4e7c-9109-e8bdb716d7f7 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:41.247788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 588/1196] 10.222.0.36 () {54 vars in 961 bytes} [Sat Oct 15 13:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:46:41.253193 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d09ff8-245d-473a-b43f-23f0799a5fa8 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:41.253636 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d09ff8-245d-473a-b43f-23f0799a5fa8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:41.253727 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d09ff8-245d-473a-b43f-23f0799a5fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:41.274315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-a5d09ff8-245d-473a-b43f-23f0799a5fa8 None None] MFA Rules not processed for user `1c6720518c03413aa8f85e4d6142fe87`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:46:41.275600 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-a5d09ff8-245d-473a-b43f-23f0799a5fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:46:41.284457 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 609/1197] 10.222.0.36 () {58 vars in 1058 bytes} [Sat Oct 15 13:46:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:46:41.292566 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c14ca-ece5-4e7c-9109-e8bdb716d7f7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:46:41.307166 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c14ca-ece5-4e7c-9109-e8bdb716d7f7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:46:41.312434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074542b8-228f-42d7-8bff-c75227f8eb4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6720518c03413aa8f85e4d6142fe87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:46:41.313071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074542b8-228f-42d7-8bff-c75227f8eb4f service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:41.313344 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074542b8-228f-42d7-8bff-c75227f8eb4f service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:41.313608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074542b8-228f-42d7-8bff-c75227f8eb4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:41.322682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074542b8-228f-42d7-8bff-c75227f8eb4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:46:41.324299 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074542b8-228f-42d7-8bff-c75227f8eb4f service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:46:41.334121 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 589/1198] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:46:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:46:48.457514 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41f32-3cd1-4478-a582-353a4432b62e None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:48.458198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41f32-3cd1-4478-a582-353a4432b62e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:48.458198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41f32-3cd1-4478-a582-353a4432b62e None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:48.459405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 610/1199] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:46:48.466833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02054bca-d595-4200-9dd6-8491714fd591 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:48.467339 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02054bca-d595-4200-9dd6-8491714fd591 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:48.467437 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02054bca-d595-4200-9dd6-8491714fd591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:48.492778 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-02054bca-d595-4200-9dd6-8491714fd591 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:46:48.494074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-02054bca-d595-4200-9dd6-8491714fd591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:46:48.503069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 590/1200] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:46:48.664424 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc41f32-3cd1-4478-a582-353a4432b62e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:46:48.681473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc41f32-3cd1-4478-a582-353a4432b62e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:46:48.687915 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132cb9e-3d23-4202-9563-a3ec4a1fd0b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:46:48.688510 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132cb9e-3d23-4202-9563-a3ec4a1fd0b1 service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:48.688787 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132cb9e-3d23-4202-9563-a3ec4a1fd0b1 service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:48.689061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132cb9e-3d23-4202-9563-a3ec4a1fd0b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:48.697671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132cb9e-3d23-4202-9563-a3ec4a1fd0b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:46:48.699247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132cb9e-3d23-4202-9563-a3ec4a1fd0b1 service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:46:48.709180 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 611/1201] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:46:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:46:49.819921 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc496d-e460-4f22-a162-8b08cbab768f None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:49.820528 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc496d-e460-4f22-a162-8b08cbab768f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:49.820528 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc496d-e460-4f22-a162-8b08cbab768f None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:49.821828 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 591/1202] 127.0.0.1 () {54 vars in 901 bytes} [Sat Oct 15 13:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:46:49.828824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fce18-3ede-46e0-a1a8-e70c625ae03b None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:49.829361 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fce18-3ede-46e0-a1a8-e70c625ae03b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:49.829361 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fce18-3ede-46e0-a1a8-e70c625ae03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:49.850541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-885fce18-3ede-46e0-a1a8-e70c625ae03b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:46:49.851797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-885fce18-3ede-46e0-a1a8-e70c625ae03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:46:49.861682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 612/1203] 127.0.0.1 () {58 vars in 998 bytes} [Sat Oct 15 13:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:46:50.013441 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc496d-e460-4f22-a162-8b08cbab768f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:46:50.028656 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc496d-e460-4f22-a162-8b08cbab768f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:46:50.034982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bb73c-68ab-43f0-bb4c-b8d749b0e2cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:46:50.035613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bb73c-68ab-43f0-bb4c-b8d749b0e2cd service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:50.035908 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bb73c-68ab-43f0-bb4c-b8d749b0e2cd service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:50.036187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bb73c-68ab-43f0-bb4c-b8d749b0e2cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:50.044773 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bb73c-68ab-43f0-bb4c-b8d749b0e2cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:46:50.046690 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bb73c-68ab-43f0-bb4c-b8d749b0e2cd service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:46:50.056244 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 592/1204] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:46:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:46:54.078066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361ae2f-eff1-4178-8d54-612bd453f888 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:54.078612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361ae2f-eff1-4178-8d54-612bd453f888 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:54.078612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361ae2f-eff1-4178-8d54-612bd453f888 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:54.079907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 613/1205] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:46:54.085544 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25035457-6647-4368-9c7e-ca0acc83a85f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:54.086168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25035457-6647-4368-9c7e-ca0acc83a85f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:54.086168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25035457-6647-4368-9c7e-ca0acc83a85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:54.108822 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-25035457-6647-4368-9c7e-ca0acc83a85f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:46:54.110205 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-25035457-6647-4368-9c7e-ca0acc83a85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:46:54.119042 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 593/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:46:54.410311 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361ae2f-eff1-4178-8d54-612bd453f888 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:46:54.412478 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2361ae2f-eff1-4178-8d54-612bd453f888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:46:54.503861 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361ae2f-eff1-4178-8d54-612bd453f888 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:46:54.508156 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b4eaf-2bf7-46ac-a270-4fda7ce7063d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:46:54.508759 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b4eaf-2bf7-46ac-a270-4fda7ce7063d service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:54.509032 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b4eaf-2bf7-46ac-a270-4fda7ce7063d service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:54.509304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b4eaf-2bf7-46ac-a270-4fda7ce7063d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:54.518610 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b4eaf-2bf7-46ac-a270-4fda7ce7063d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:46:54.520064 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b4eaf-2bf7-46ac-a270-4fda7ce7063d service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:46:54.529846 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 614/1207] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:46:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:46:56.694226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a4d86-be0c-4667-a463-da65380c7ecf None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:56.694833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a4d86-be0c-4667-a463-da65380c7ecf None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:56.694833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a4d86-be0c-4667-a463-da65380c7ecf None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:56.696202 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 594/1208] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:46:56.701920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67ab20-484e-4830-863b-449c461abc9f None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:56.702486 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67ab20-484e-4830-863b-449c461abc9f None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:56.702486 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67ab20-484e-4830-863b-449c461abc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:56.722829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-1d67ab20-484e-4830-863b-449c461abc9f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:46:56.724056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1d67ab20-484e-4830-863b-449c461abc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:46:56.732937 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 615/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:46:57.022333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a4d86-be0c-4667-a463-da65380c7ecf None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:46:57.034452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a4d86-be0c-4667-a463-da65380c7ecf None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:46:57.038274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c5e44-06eb-4631-9b7c-134cc0093cbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:46:57.038879 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c5e44-06eb-4631-9b7c-134cc0093cbe service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:46:57.039154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c5e44-06eb-4631-9b7c-134cc0093cbe service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:46:57.039423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c5e44-06eb-4631-9b7c-134cc0093cbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:46:57.049564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c5e44-06eb-4631-9b7c-134cc0093cbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:46:57.051008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c5e44-06eb-4631-9b7c-134cc0093cbe service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:46:57.061023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 595/1210] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:46:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:47:00.755561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458d46c-b18c-48ea-9912-6cdbfe75318a None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:47:00.756236 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458d46c-b18c-48ea-9912-6cdbfe75318a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:47:00.756236 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458d46c-b18c-48ea-9912-6cdbfe75318a None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:47:00.757502 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 616/1211] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:47:00.763912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8647c-99e6-4ca1-ad31-9d5550ef8fdc None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:47:00.764762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8647c-99e6-4ca1-ad31-9d5550ef8fdc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:47:00.764954 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8647c-99e6-4ca1-ad31-9d5550ef8fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:47:00.773019 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=81 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:47:00.786742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-67d8647c-99e6-4ca1-ad31-9d5550ef8fdc None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:47:00.787980 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-67d8647c-99e6-4ca1-ad31-9d5550ef8fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:47:00.797484 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 596/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:47:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:47:01.103188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458d46c-b18c-48ea-9912-6cdbfe75318a None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:47:01.105688 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6458d46c-b18c-48ea-9912-6cdbfe75318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:47:01.179440 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:47:01.195307 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458d46c-b18c-48ea-9912-6cdbfe75318a None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:47:01.199816 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296cf40b-d532-4b60-989f-e5177dfc7466 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:47:01.200454 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cf40b-d532-4b60-989f-e5177dfc7466 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:47:01.200730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cf40b-d532-4b60-989f-e5177dfc7466 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:47:01.200996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cf40b-d532-4b60-989f-e5177dfc7466 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:47:01.210494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296cf40b-d532-4b60-989f-e5177dfc7466 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:47:01.211950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296cf40b-d532-4b60-989f-e5177dfc7466 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:47:01.222221 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 617/1213] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:47:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:47:03.390815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a31bd9-e2f4-4eca-881b-197b4e004887 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:47:03.391420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a31bd9-e2f4-4eca-881b-197b4e004887 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:47:03.391420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a31bd9-e2f4-4eca-881b-197b4e004887 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:47:03.392817 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 597/1214] 127.0.0.1 () {54 vars in 917 bytes} [Sat Oct 15 13:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:47:03.398221 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75730c2b-47c1-491e-8550-44007663f238 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:47:03.398660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75730c2b-47c1-491e-8550-44007663f238 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:47:03.398767 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75730c2b-47c1-491e-8550-44007663f238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:47:03.420554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-75730c2b-47c1-491e-8550-44007663f238 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:47:03.421751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-75730c2b-47c1-491e-8550-44007663f238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:47:03.430328 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 618/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Oct 15 13:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:47:03.742435 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a31bd9-e2f4-4eca-881b-197b4e004887 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:47:03.745551 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c3a31bd9-e2f4-4eca-881b-197b4e004887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:47:03.770235 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a31bd9-e2f4-4eca-881b-197b4e004887 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:47:03.776650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001dc4e-2013-4ddd-8134-4b01cd4fa221 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:47:03.777453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001dc4e-2013-4ddd-8134-4b01cd4fa221 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:47:03.777756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001dc4e-2013-4ddd-8134-4b01cd4fa221 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:47:03.778046 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001dc4e-2013-4ddd-8134-4b01cd4fa221 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:47:03.792108 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001dc4e-2013-4ddd-8134-4b01cd4fa221 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:47:03.793563 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001dc4e-2013-4ddd-8134-4b01cd4fa221 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:47:03.803505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 598/1216] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:47:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:47:13.414870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=11 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:47:13.799111 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=8 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:27.670443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d63d5-bd7e-4930-9442-dff9a2b21cd3 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:27.671520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d63d5-bd7e-4930-9442-dff9a2b21cd3 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:27.671990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d63d5-bd7e-4930-9442-dff9a2b21cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:27.706937 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-052d63d5-bd7e-4930-9442-dff9a2b21cd3 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:27.708298 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-052d63d5-bd7e-4930-9442-dff9a2b21cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:27.725225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 619/1217] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:27.735293 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331fa94-cd58-46e5-9caa-c5c3be1b7c05 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:27.736508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331fa94-cd58-46e5-9caa-c5c3be1b7c05 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:27.737379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331fa94-cd58-46e5-9caa-c5c3be1b7c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:27.738795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b570551-3380-45ec-a434-b75f7f03e4c8 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:27.739219 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b570551-3380-45ec-a434-b75f7f03e4c8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:27.739411 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b570551-3380-45ec-a434-b75f7f03e4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:27.770818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-0b570551-3380-45ec-a434-b75f7f03e4c8 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:27.772815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9331fa94-cd58-46e5-9caa-c5c3be1b7c05 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:27.773245 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0b570551-3380-45ec-a434-b75f7f03e4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:27.774321 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9331fa94-cd58-46e5-9caa-c5c3be1b7c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:27.799769 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 620/1218] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:48:27.808527 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b570551-3380-45ec-a434-b75f7f03e4c8 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:27.814148 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 599/1219] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:48:27.831127 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcc345-f9bd-4eaf-8db8-a1ac8303d6d4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:27.831492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcc345-f9bd-4eaf-8db8-a1ac8303d6d4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:27.831907 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcc345-f9bd-4eaf-8db8-a1ac8303d6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:27.846729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b570551-3380-45ec-a434-b75f7f03e4c8 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:27.861866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e7ae6-e697-47d9-8ec3-08c7e402fe76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:27.862881 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e7ae6-e697-47d9-8ec3-08c7e402fe76 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:27.863347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e7ae6-e697-47d9-8ec3-08c7e402fe76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:27.863798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e7ae6-e697-47d9-8ec3-08c7e402fe76 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:27.864950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e7ae6-e697-47d9-8ec3-08c7e402fe76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:48:27.865461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e7ae6-e697-47d9-8ec3-08c7e402fe76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:27.885981 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-aedcc345-f9bd-4eaf-8db8-a1ac8303d6d4 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:27.887223 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-aedcc345-f9bd-4eaf-8db8-a1ac8303d6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:27.891751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e7ae6-e697-47d9-8ec3-08c7e402fe76 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:27.926274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 600/1220] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:27.927586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 621/1221] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:48:27.930046 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b63a8e-4a91-4962-a34c-f76685d2b5fa None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:27.930328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b63a8e-4a91-4962-a34c-f76685d2b5fa None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:27.930598 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b63a8e-4a91-4962-a34c-f76685d2b5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:27.933597 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e7ae6-e697-47d9-8ec3-08c7e402fe76 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:27.964260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e7ae6-e697-47d9-8ec3-08c7e402fe76 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:27.974611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b6b63a8e-4a91-4962-a34c-f76685d2b5fa None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:27.975840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b6b63a8e-4a91-4962-a34c-f76685d2b5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:27.982973 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d33d31-e290-42b8-9898-69c204358717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:27.984022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d33d31-e290-42b8-9898-69c204358717 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:27.984508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d33d31-e290-42b8-9898-69c204358717 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:27.984991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d33d31-e290-42b8-9898-69c204358717 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:27.986115 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d33d31-e290-42b8-9898-69c204358717 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:48:27.986625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d33d31-e290-42b8-9898-69c204358717 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:27.998017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d33d31-e290-42b8-9898-69c204358717 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.001904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 601/1222] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:48:28.002584 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b63a8e-4a91-4962-a34c-f76685d2b5fa None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.025329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b63a8e-4a91-4962-a34c-f76685d2b5fa None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.034039 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 622/1223] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:48:28.036307 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447a2a7-cad6-45db-b916-4af3044b72ac None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.036857 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447a2a7-cad6-45db-b916-4af3044b72ac None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.037400 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447a2a7-cad6-45db-b916-4af3044b72ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.039867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8b850-3970-4611-8a26-753aa0ffd2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.040542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8b850-3970-4611-8a26-753aa0ffd2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.041070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8b850-3970-4611-8a26-753aa0ffd2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.041399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8b850-3970-4611-8a26-753aa0ffd2e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.042037 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8b850-3970-4611-8a26-753aa0ffd2e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.044006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8b850-3970-4611-8a26-753aa0ffd2e9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.110704 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-7447a2a7-cad6-45db-b916-4af3044b72ac None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:28.118977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7447a2a7-cad6-45db-b916-4af3044b72ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:28.140815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 623/1224] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:28.142258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447a2a7-cad6-45db-b916-4af3044b72ac None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.147146 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 602/1225] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:28.148427 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43dc62-d04c-4b22-aaef-0e6c6b472538 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.148727 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43dc62-d04c-4b22-aaef-0e6c6b472538 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.149040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43dc62-d04c-4b22-aaef-0e6c6b472538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.178539 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447a2a7-cad6-45db-b916-4af3044b72ac None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.192967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-4c43dc62-d04c-4b22-aaef-0e6c6b472538 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:28.196257 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4c43dc62-d04c-4b22-aaef-0e6c6b472538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:28.274871 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 603/1226] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:28.275292 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43dc62-d04c-4b22-aaef-0e6c6b472538 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.294677 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43dc62-d04c-4b22-aaef-0e6c6b472538 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.300608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727a056-079b-4398-830d-e3594d18a5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.301236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727a056-079b-4398-830d-e3594d18a5fa admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.301506 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727a056-079b-4398-830d-e3594d18a5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.301768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727a056-079b-4398-830d-e3594d18a5fa admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.302416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727a056-079b-4398-830d-e3594d18a5fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:48:28.302718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727a056-079b-4398-830d-e3594d18a5fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.303761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-8727a056-079b-4398-830d-e3594d18a5fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:48:28.303761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Oct 15 13:48:28.303761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: Oct 15 13:48:28.304729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727a056-079b-4398-830d-e3594d18a5fa admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.310272 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4501e0-02aa-4d5b-8df1-31799d34ca3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.310907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4501e0-02aa-4d5b-8df1-31799d34ca3c admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.311182 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4501e0-02aa-4d5b-8df1-31799d34ca3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.311455 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4501e0-02aa-4d5b-8df1-31799d34ca3c admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.312129 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4501e0-02aa-4d5b-8df1-31799d34ca3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.313170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: WARNING py.warnings [None req-6e4501e0-02aa-4d5b-8df1-31799d34ca3c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:48:28.313170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: warnings.warn(msg) Oct 15 13:48:28.313170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: Oct 15 13:48:28.314361 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4501e0-02aa-4d5b-8df1-31799d34ca3c admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.324154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 604/1227] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:48:28.324154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727a056-079b-4398-830d-e3594d18a5fa admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.330373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727a056-079b-4398-830d-e3594d18a5fa admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.335448 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff2cfd-30ef-4231-aa21-e647562bbe46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.336156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff2cfd-30ef-4231-aa21-e647562bbe46 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.336437 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff2cfd-30ef-4231-aa21-e647562bbe46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.336765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff2cfd-30ef-4231-aa21-e647562bbe46 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.337445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff2cfd-30ef-4231-aa21-e647562bbe46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.338481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-b3ff2cfd-30ef-4231-aa21-e647562bbe46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:48:28.338481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Oct 15 13:48:28.338481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: Oct 15 13:48:28.339699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff2cfd-30ef-4231-aa21-e647562bbe46 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.346766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 624/1228] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/projects => generated 375 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:28.348106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c09deb-5159-487d-b322-0ce4765471f2 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.348674 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c09deb-5159-487d-b322-0ce4765471f2 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.348674 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c09deb-5159-487d-b322-0ce4765471f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.365304 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 605/1229] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:28.366776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52f2c4-c002-422b-838b-21b082015499 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.367053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52f2c4-c002-422b-838b-21b082015499 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.367315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52f2c4-c002-422b-838b-21b082015499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.374298 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-04c09deb-5159-487d-b322-0ce4765471f2 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:28.375530 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-04c09deb-5159-487d-b322-0ce4765471f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:28.387830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ea52f2c4-c002-422b-838b-21b082015499 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:28.389331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ea52f2c4-c002-422b-838b-21b082015499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:28.446313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 625/1230] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:28.446313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320faae3-b04f-4096-bd41-08277d5fb367 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.446313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320faae3-b04f-4096-bd41-08277d5fb367 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.446313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320faae3-b04f-4096-bd41-08277d5fb367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.463282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 606/1231] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:28.465132 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb65b2f-9f54-41c9-8af0-ce4875ce64ca None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.465557 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb65b2f-9f54-41c9-8af0-ce4875ce64ca None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.465997 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb65b2f-9f54-41c9-8af0-ce4875ce64ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.466927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-320faae3-b04f-4096-bd41-08277d5fb367 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:28.468209 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-320faae3-b04f-4096-bd41-08277d5fb367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:28.476856 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 626/1232] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:28.478524 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6d32b-087b-4fd3-b27d-3bf025ef0a92 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.478808 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6d32b-087b-4fd3-b27d-3bf025ef0a92 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.479072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6d32b-087b-4fd3-b27d-3bf025ef0a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.491309 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-1eb65b2f-9f54-41c9-8af0-ce4875ce64ca None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:28.492518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1eb65b2f-9f54-41c9-8af0-ce4875ce64ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:28.500957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 607/1233] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:28.501319 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-94f6d32b-087b-4fd3-b27d-3bf025ef0a92 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:28.501784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb65b2f-9f54-41c9-8af0-ce4875ce64ca None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.502594 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-94f6d32b-087b-4fd3-b27d-3bf025ef0a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:28.511759 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 627/1234] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:28.512179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb65b2f-9f54-41c9-8af0-ce4875ce64ca None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.512710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6d32b-087b-4fd3-b27d-3bf025ef0a92 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.517366 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b695cf-50b6-4167-86c0-9d79797197d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.518108 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b695cf-50b6-4167-86c0-9d79797197d7 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.518284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b695cf-50b6-4167-86c0-9d79797197d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.518559 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b695cf-50b6-4167-86c0-9d79797197d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.519138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b695cf-50b6-4167-86c0-9d79797197d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.521129 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b695cf-50b6-4167-86c0-9d79797197d7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.524586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6d32b-087b-4fd3-b27d-3bf025ef0a92 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.529536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08155a6d-90d2-4bc7-ae27-f0a0e8945259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.530167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08155a6d-90d2-4bc7-ae27-f0a0e8945259 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.530448 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08155a6d-90d2-4bc7-ae27-f0a0e8945259 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.530905 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08155a6d-90d2-4bc7-ae27-f0a0e8945259 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.531658 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08155a6d-90d2-4bc7-ae27-f0a0e8945259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.533729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08155a6d-90d2-4bc7-ae27-f0a0e8945259 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.534821 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:28.558590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 628/1235] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/users => generated 288 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:28.559453 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08155a6d-90d2-4bc7-ae27-f0a0e8945259 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.560626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 608/1236] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:28.563474 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ee7f1-f7c5-4002-9599-217c0966aa16 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.563781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ee7f1-f7c5-4002-9599-217c0966aa16 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.564312 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ee7f1-f7c5-4002-9599-217c0966aa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.574796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08155a6d-90d2-4bc7-ae27-f0a0e8945259 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.580534 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:28.589474 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0b8ee7f1-f7c5-4002-9599-217c0966aa16 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:28.591530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0b8ee7f1-f7c5-4002-9599-217c0966aa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:28.644600 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19e2a5-9119-4152-9702-6edf2280570c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.645305 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19e2a5-9119-4152-9702-6edf2280570c admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.645709 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19e2a5-9119-4152-9702-6edf2280570c admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.646040 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19e2a5-9119-4152-9702-6edf2280570c admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.646942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19e2a5-9119-4152-9702-6edf2280570c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.649139 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19e2a5-9119-4152-9702-6edf2280570c admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.659189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 609/1237] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:28.660872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377370e0-2f1d-4e3c-818c-5da768c0bdda None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.661304 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377370e0-2f1d-4e3c-818c-5da768c0bdda None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.661750 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377370e0-2f1d-4e3c-818c-5da768c0bdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.662884 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 629/1238] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:28.666665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daacc35-7779-49e1-8f22-a124b980a804 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.667083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daacc35-7779-49e1-8f22-a124b980a804 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.667492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daacc35-7779-49e1-8f22-a124b980a804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.683570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-377370e0-2f1d-4e3c-818c-5da768c0bdda None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:28.684898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-377370e0-2f1d-4e3c-818c-5da768c0bdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:28.687685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-7daacc35-7779-49e1-8f22-a124b980a804 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:28.688852 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7daacc35-7779-49e1-8f22-a124b980a804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:28.693484 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 610/1239] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:28.694116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377370e0-2f1d-4e3c-818c-5da768c0bdda None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.698986 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 630/1240] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:28.700195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a02a3-6edf-4124-8b3f-274e07d25d03 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.700531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a02a3-6edf-4124-8b3f-274e07d25d03 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.701640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a02a3-6edf-4124-8b3f-274e07d25d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.706098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377370e0-2f1d-4e3c-818c-5da768c0bdda None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.712975 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ffcdf-5da0-42e2-82c7-8d394d20bca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.713592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ffcdf-5da0-42e2-82c7-8d394d20bca9 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.713863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ffcdf-5da0-42e2-82c7-8d394d20bca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.714123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ffcdf-5da0-42e2-82c7-8d394d20bca9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.714762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ffcdf-5da0-42e2-82c7-8d394d20bca9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:48:28.715068 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ffcdf-5da0-42e2-82c7-8d394d20bca9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.717599 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ffcdf-5da0-42e2-82c7-8d394d20bca9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.721803 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-2e4a02a3-6edf-4124-8b3f-274e07d25d03 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:28.723037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2e4a02a3-6edf-4124-8b3f-274e07d25d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:28.730836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 611/1241] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:48:28.731742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ffcdf-5da0-42e2-82c7-8d394d20bca9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.737107 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 631/1242] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:28.738186 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a02a3-6edf-4124-8b3f-274e07d25d03 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.743245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ffcdf-5da0-42e2-82c7-8d394d20bca9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.747999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e2d08-9d04-44d1-86f0-21497a9cf323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.748621 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e2d08-9d04-44d1-86f0-21497a9cf323 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.748895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e2d08-9d04-44d1-86f0-21497a9cf323 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.749333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a02a3-6edf-4124-8b3f-274e07d25d03 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.749539 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e2d08-9d04-44d1-86f0-21497a9cf323 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.749996 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e2d08-9d04-44d1-86f0-21497a9cf323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.751617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e2d08-9d04-44d1-86f0-21497a9cf323 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.754219 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cffbb-5e91-429b-b0a0-ac6196aa6275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.754834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cffbb-5e91-429b-b0a0-ac6196aa6275 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.755106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cffbb-5e91-429b-b0a0-ac6196aa6275 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.755372 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cffbb-5e91-429b-b0a0-ac6196aa6275 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.756068 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652cffbb-5e91-429b-b0a0-ac6196aa6275 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.757987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652cffbb-5e91-429b-b0a0-ac6196aa6275 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.763647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 612/1243] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:28.764769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e2d08-9d04-44d1-86f0-21497a9cf323 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.776320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e2d08-9d04-44d1-86f0-21497a9cf323 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.782143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b2e84-7d07-45ee-878a-2810005b0414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.782795 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b2e84-7d07-45ee-878a-2810005b0414 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.783066 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b2e84-7d07-45ee-878a-2810005b0414 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.783333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b2e84-7d07-45ee-878a-2810005b0414 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.784003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b2e84-7d07-45ee-878a-2810005b0414 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:48:28.784300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b2e84-7d07-45ee-878a-2810005b0414 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.785171 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 632/1244] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/users => generated 290 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:28.786158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b2e84-7d07-45ee-878a-2810005b0414 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.789421 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cffbb-5e91-429b-b0a0-ac6196aa6275 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.811404 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 613/1245] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:48:28.811404 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b2e84-7d07-45ee-878a-2810005b0414 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.811867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cffbb-5e91-429b-b0a0-ac6196aa6275 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.811867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0c3c2-2a4e-42b9-9e67-966576215877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.812247 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b2e84-7d07-45ee-878a-2810005b0414 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.812967 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0c3c2-2a4e-42b9-9e67-966576215877 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.813505 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0c3c2-2a4e-42b9-9e67-966576215877 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.814105 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0c3c2-2a4e-42b9-9e67-966576215877 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.815151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0c3c2-2a4e-42b9-9e67-966576215877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.817280 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0c3c2-2a4e-42b9-9e67-966576215877 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.817977 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d43401-6720-47bb-828c-090d85ecc0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.818612 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d43401-6720-47bb-828c-090d85ecc0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.818868 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d43401-6720-47bb-828c-090d85ecc0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.819162 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d43401-6720-47bb-828c-090d85ecc0e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.819833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d43401-6720-47bb-828c-090d85ecc0e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:48:28.820216 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d43401-6720-47bb-828c-090d85ecc0e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.822083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d43401-6720-47bb-828c-090d85ecc0e3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.831433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 633/1246] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:28.831433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0c3c2-2a4e-42b9-9e67-966576215877 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.838998 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 614/1247] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:48:28.840005 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d43401-6720-47bb-828c-090d85ecc0e3 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.847307 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0c3c2-2a4e-42b9-9e67-966576215877 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.854238 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf48e40-39aa-46bb-9f33-748683cc585c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.854965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf48e40-39aa-46bb-9f33-748683cc585c admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.855386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf48e40-39aa-46bb-9f33-748683cc585c admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.855755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf48e40-39aa-46bb-9f33-748683cc585c admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.856583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf48e40-39aa-46bb-9f33-748683cc585c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:48:28.857354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf48e40-39aa-46bb-9f33-748683cc585c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.858164 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d43401-6720-47bb-828c-090d85ecc0e3 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.859091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf48e40-39aa-46bb-9f33-748683cc585c admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.862951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e62bb-04e6-4e89-ba77-3eecf66bce8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.864332 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e62bb-04e6-4e89-ba77-3eecf66bce8a admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.864610 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e62bb-04e6-4e89-ba77-3eecf66bce8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.864876 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e62bb-04e6-4e89-ba77-3eecf66bce8a admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.865518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e62bb-04e6-4e89-ba77-3eecf66bce8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:48:28.865817 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e62bb-04e6-4e89-ba77-3eecf66bce8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.867450 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e62bb-04e6-4e89-ba77-3eecf66bce8a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.873754 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 634/1248] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:48:28.875446 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf48e40-39aa-46bb-9f33-748683cc585c admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.894222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 615/1249] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:48:28.894222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e62bb-04e6-4e89-ba77-3eecf66bce8a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.894559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf48e40-39aa-46bb-9f33-748683cc585c admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.899576 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e5f63-213d-40e3-8c2c-e093cc6a7711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.900279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e5f63-213d-40e3-8c2c-e093cc6a7711 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.900559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e5f63-213d-40e3-8c2c-e093cc6a7711 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.900830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e5f63-213d-40e3-8c2c-e093cc6a7711 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.901473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e5f63-213d-40e3-8c2c-e093cc6a7711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.903441 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e5f63-213d-40e3-8c2c-e093cc6a7711 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.907630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e62bb-04e6-4e89-ba77-3eecf66bce8a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.912983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933e1a2-f157-4b25-98e1-01a5e4782bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:28.913667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933e1a2-f157-4b25-98e1-01a5e4782bfa admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:28.914464 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933e1a2-f157-4b25-98e1-01a5e4782bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:28.914464 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933e1a2-f157-4b25-98e1-01a5e4782bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:28.915032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933e1a2-f157-4b25-98e1-01a5e4782bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:28.916885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933e1a2-f157-4b25-98e1-01a5e4782bfa admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:28.930000 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 616/1250] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:28.933508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933e1a2-f157-4b25-98e1-01a5e4782bfa admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:28.941518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933e1a2-f157-4b25-98e1-01a5e4782bfa admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:28.941814 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 635/1251] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:28.942620 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e5f63-213d-40e3-8c2c-e093cc6a7711 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.010762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59fdd6-a766-4418-857f-b328707d2306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.011618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59fdd6-a766-4418-857f-b328707d2306 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.011924 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59fdd6-a766-4418-857f-b328707d2306 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.012310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59fdd6-a766-4418-857f-b328707d2306 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.013407 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59fdd6-a766-4418-857f-b328707d2306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.016236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59fdd6-a766-4418-857f-b328707d2306 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.017838 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e5f63-213d-40e3-8c2c-e093cc6a7711 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.025249 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586428c4-4356-4663-8347-96ca107a742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.025881 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586428c4-4356-4663-8347-96ca107a742f admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.026152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586428c4-4356-4663-8347-96ca107a742f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.026570 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586428c4-4356-4663-8347-96ca107a742f admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.027241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586428c4-4356-4663-8347-96ca107a742f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.029578 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586428c4-4356-4663-8347-96ca107a742f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.033178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 617/1252] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:29.033178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59fdd6-a766-4418-857f-b328707d2306 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.042660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59fdd6-a766-4418-857f-b328707d2306 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.048598 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ceb51-3eb1-471b-908d-eab0e77743d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.049617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ceb51-3eb1-471b-908d-eab0e77743d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.050001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ceb51-3eb1-471b-908d-eab0e77743d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.050410 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ceb51-3eb1-471b-908d-eab0e77743d9 admin admin] PATH_INFO: `/v3/projects/080f5405f5a44718858ccbc99dc49680/users/8b3f059dc7e04446a8f04da22761c30f/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.053374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:29.063815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 636/1253] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:28 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:29.068388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586428c4-4356-4663-8347-96ca107a742f admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.073431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ceb51-3eb1-471b-908d-eab0e77743d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=080f5405f5a44718858ccbc99dc49680, user_id=8b3f059dc7e04446a8f04da22761c30f, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.077682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ceb51-3eb1-471b-908d-eab0e77743d9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.084982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:29.093572 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 618/1254] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:29 2022] PUT /identity/v3/projects/080f5405f5a44718858ccbc99dc49680/users/8b3f059dc7e04446a8f04da22761c30f/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:29.094276 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ceb51-3eb1-471b-908d-eab0e77743d9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.155258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586428c4-4356-4663-8347-96ca107a742f admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.185240 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ceb51-3eb1-471b-908d-eab0e77743d9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.192323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63070785-8408-4058-b056-b7c42ab00f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.193300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63070785-8408-4058-b056-b7c42ab00f39 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.193747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63070785-8408-4058-b056-b7c42ab00f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.194133 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63070785-8408-4058-b056-b7c42ab00f39 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.194919 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63070785-8408-4058-b056-b7c42ab00f39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.197325 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63070785-8408-4058-b056-b7c42ab00f39 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.232280 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 619/1255] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:29 2022] POST /identity/v3/projects => generated 405 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:29.233130 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63070785-8408-4058-b056-b7c42ab00f39 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.235760 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb5118-84b9-458e-a80d-10dee468fb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.236544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb5118-84b9-458e-a80d-10dee468fb4f admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.236897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb5118-84b9-458e-a80d-10dee468fb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.237174 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb5118-84b9-458e-a80d-10dee468fb4f admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.237841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb5118-84b9-458e-a80d-10dee468fb4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.239929 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb5118-84b9-458e-a80d-10dee468fb4f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.274431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 637/1256] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:29 2022] POST /identity/v3/projects => generated 393 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:29.275166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb5118-84b9-458e-a80d-10dee468fb4f admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.314488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63070785-8408-4058-b056-b7c42ab00f39 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.343074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb5118-84b9-458e-a80d-10dee468fb4f admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.348222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff98a4c-d8f2-478a-b49e-4b1d1e41bbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.348907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff98a4c-d8f2-478a-b49e-4b1d1e41bbbb admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.349176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff98a4c-d8f2-478a-b49e-4b1d1e41bbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.349442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff98a4c-d8f2-478a-b49e-4b1d1e41bbbb admin admin] PATH_INFO: `/v3/projects/35f8b5024a194f70a9f7689ac44ab07b/users/583cce1547144e11978f20a3193ceba4/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.366404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff98a4c-d8f2-478a-b49e-4b1d1e41bbbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f8b5024a194f70a9f7689ac44ab07b, user_id=583cce1547144e11978f20a3193ceba4, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.369125 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff98a4c-d8f2-478a-b49e-4b1d1e41bbbb admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.378763 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0189858-e1bd-4c53-93d8-909bad54e077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.379406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0189858-e1bd-4c53-93d8-909bad54e077 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.379681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0189858-e1bd-4c53-93d8-909bad54e077 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.379989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0189858-e1bd-4c53-93d8-909bad54e077 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.380637 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0189858-e1bd-4c53-93d8-909bad54e077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.382578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0189858-e1bd-4c53-93d8-909bad54e077 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.386020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 638/1257] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:29 2022] PUT /identity/v3/projects/35f8b5024a194f70a9f7689ac44ab07b/users/583cce1547144e11978f20a3193ceba4/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:29.387479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff98a4c-d8f2-478a-b49e-4b1d1e41bbbb admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.414440 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:29.423714 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 620/1258] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:29 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:29.425639 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0189858-e1bd-4c53-93d8-909bad54e077 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.429771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:29.471678 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff98a4c-d8f2-478a-b49e-4b1d1e41bbbb admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.505130 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0189858-e1bd-4c53-93d8-909bad54e077 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.513376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d82931-4b0f-45c6-968e-8fb3101850f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.514125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d82931-4b0f-45c6-968e-8fb3101850f9 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.514542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d82931-4b0f-45c6-968e-8fb3101850f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.514967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d82931-4b0f-45c6-968e-8fb3101850f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.516065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d82931-4b0f-45c6-968e-8fb3101850f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.518817 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d82931-4b0f-45c6-968e-8fb3101850f9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.534405 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 621/1259] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:29.535821 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d82931-4b0f-45c6-968e-8fb3101850f9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.545144 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42408731-9147-41fc-9cf0-ec2aee48030a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.545774 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408731-9147-41fc-9cf0-ec2aee48030a admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.546052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408731-9147-41fc-9cf0-ec2aee48030a admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.546321 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408731-9147-41fc-9cf0-ec2aee48030a admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.546971 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42408731-9147-41fc-9cf0-ec2aee48030a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.548285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d82931-4b0f-45c6-968e-8fb3101850f9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.548982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42408731-9147-41fc-9cf0-ec2aee48030a admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.553178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b130a0-1d07-4af0-af37-1c175aef370c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.553802 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b130a0-1d07-4af0-af37-1c175aef370c admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.554073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b130a0-1d07-4af0-af37-1c175aef370c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.554347 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b130a0-1d07-4af0-af37-1c175aef370c admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.554978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b130a0-1d07-4af0-af37-1c175aef370c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.557151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b130a0-1d07-4af0-af37-1c175aef370c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.576339 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 639/1260] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:29 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:29.581232 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4045086-9b5b-4abe-9b55-2020a2a51e61 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.581232 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4045086-9b5b-4abe-9b55-2020a2a51e61 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.581232 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4045086-9b5b-4abe-9b55-2020a2a51e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.591158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 622/1261] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:29 2022] POST /identity/v3/users => generated 293 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:29.591158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b130a0-1d07-4af0-af37-1c175aef370c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.600839 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b130a0-1d07-4af0-af37-1c175aef370c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.607165 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f721c8-33ad-402d-96fc-e62212c61bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.607945 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f721c8-33ad-402d-96fc-e62212c61bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.608198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-e4045086-9b5b-4abe-9b55-2020a2a51e61 None None] MFA Rules not processed for user `8b3f059dc7e04446a8f04da22761c30f`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:29.608387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f721c8-33ad-402d-96fc-e62212c61bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.608734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f721c8-33ad-402d-96fc-e62212c61bb4 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.609455 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e4045086-9b5b-4abe-9b55-2020a2a51e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:29.609640 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f721c8-33ad-402d-96fc-e62212c61bb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.611959 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f721c8-33ad-402d-96fc-e62212c61bb4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.641553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 623/1262] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:29 2022] POST /identity/v3/users => generated 303 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:29.642804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f721c8-33ad-402d-96fc-e62212c61bb4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.655312 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f721c8-33ad-402d-96fc-e62212c61bb4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.660201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8661fc7-837c-4684-ae9f-079bc3bf42c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.660830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8661fc7-837c-4684-ae9f-079bc3bf42c5 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.661141 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8661fc7-837c-4684-ae9f-079bc3bf42c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.661411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8661fc7-837c-4684-ae9f-079bc3bf42c5 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.662021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8661fc7-837c-4684-ae9f-079bc3bf42c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.664023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8661fc7-837c-4684-ae9f-079bc3bf42c5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.688217 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 624/1263] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:29 2022] POST /identity/v3/users => generated 297 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:29.689551 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3296cfd-cfc9-425f-a89e-d24adb30414a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.689854 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3296cfd-cfc9-425f-a89e-d24adb30414a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.690117 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3296cfd-cfc9-425f-a89e-d24adb30414a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.694551 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 640/1264] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:29.695410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4045086-9b5b-4abe-9b55-2020a2a51e61 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.712582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4045086-9b5b-4abe-9b55-2020a2a51e61 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.720476 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29ae44-5ff2-4a32-8fc6-32aedee99379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.720696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b3296cfd-cfc9-425f-a89e-d24adb30414a None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:29.721372 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29ae44-5ff2-4a32-8fc6-32aedee99379 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.721716 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b3296cfd-cfc9-425f-a89e-d24adb30414a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:29.721913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29ae44-5ff2-4a32-8fc6-32aedee99379 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.722222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29ae44-5ff2-4a32-8fc6-32aedee99379 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.723064 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29ae44-5ff2-4a32-8fc6-32aedee99379 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.726937 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29ae44-5ff2-4a32-8fc6-32aedee99379 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.731948 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 625/1265] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:29.743621 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3296cfd-cfc9-425f-a89e-d24adb30414a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.755379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3296cfd-cfc9-425f-a89e-d24adb30414a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.761223 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b110794-4cca-4485-85ef-eda9bf1981a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.761984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b110794-4cca-4485-85ef-eda9bf1981a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.762128 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b110794-4cca-4485-85ef-eda9bf1981a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.762431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b110794-4cca-4485-85ef-eda9bf1981a9 admin admin] PATH_INFO: `/v3/projects/984a02b8dfd446879438d29b5efdd996/users/bc29734859a5460faee189b1726b50b4/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.763447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 641/1266] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:29 2022] POST /identity/v3/users => generated 294 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:29.765038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29ae44-5ff2-4a32-8fc6-32aedee99379 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.781522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29ae44-5ff2-4a32-8fc6-32aedee99379 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.788295 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2cdc6-5146-43dc-9632-34c512fff37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.788843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2cdc6-5146-43dc-9632-34c512fff37c admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.788970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b110794-4cca-4485-85ef-eda9bf1981a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=984a02b8dfd446879438d29b5efdd996, user_id=bc29734859a5460faee189b1726b50b4, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.789122 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2cdc6-5146-43dc-9632-34c512fff37c admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.789447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2cdc6-5146-43dc-9632-34c512fff37c admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.790197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2cdc6-5146-43dc-9632-34c512fff37c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.791990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2cdc6-5146-43dc-9632-34c512fff37c admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.792248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b110794-4cca-4485-85ef-eda9bf1981a9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.806377 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 642/1267] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:29.807929 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2cdc6-5146-43dc-9632-34c512fff37c admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.813888 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 626/1268] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:29 2022] PUT /identity/v3/projects/984a02b8dfd446879438d29b5efdd996/users/bc29734859a5460faee189b1726b50b4/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:29.815327 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b110794-4cca-4485-85ef-eda9bf1981a9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.820993 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2cdc6-5146-43dc-9632-34c512fff37c admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.855936 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:29.906116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3453b-0375-4b16-87bb-f431e8fd613f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.906786 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3453b-0375-4b16-87bb-f431e8fd613f admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.907065 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3453b-0375-4b16-87bb-f431e8fd613f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.907345 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3453b-0375-4b16-87bb-f431e8fd613f admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.908048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3453b-0375-4b16-87bb-f431e8fd613f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.909720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3453b-0375-4b16-87bb-f431e8fd613f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.910806 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b110794-4cca-4485-85ef-eda9bf1981a9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.915570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10070d96-81a1-4bdd-9105-3aece1b26843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.916226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10070d96-81a1-4bdd-9105-3aece1b26843 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.916495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10070d96-81a1-4bdd-9105-3aece1b26843 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.916765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10070d96-81a1-4bdd-9105-3aece1b26843 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.917406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10070d96-81a1-4bdd-9105-3aece1b26843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.919068 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10070d96-81a1-4bdd-9105-3aece1b26843 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.921367 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 643/1269] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:29.922254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3453b-0375-4b16-87bb-f431e8fd613f admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.932654 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3453b-0375-4b16-87bb-f431e8fd613f admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.936148 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 627/1270] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:29.937319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b70e2a-e8f6-4708-a179-9d98c6475405 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.937602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b70e2a-e8f6-4708-a179-9d98c6475405 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.937872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b70e2a-e8f6-4708-a179-9d98c6475405 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.939200 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 628/1271] 10.222.0.36 () {54 vars in 961 bytes} [Sat Oct 15 13:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:48:29.940099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b70e2a-e8f6-4708-a179-9d98c6475405 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.940833 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61cbcb9-257b-4769-8d3c-6e28d9150b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.941473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cbcb9-257b-4769-8d3c-6e28d9150b88 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.941745 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cbcb9-257b-4769-8d3c-6e28d9150b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.942022 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d3b70e2a-e8f6-4708-a179-9d98c6475405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:29.942181 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cbcb9-257b-4769-8d3c-6e28d9150b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.942823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61cbcb9-257b-4769-8d3c-6e28d9150b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.944532 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61cbcb9-257b-4769-8d3c-6e28d9150b88 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.955352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:29.956521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 644/1272] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:29.957420 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61cbcb9-257b-4769-8d3c-6e28d9150b88 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:29.969869 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61cbcb9-257b-4769-8d3c-6e28d9150b88 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:29.976138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bb035-4616-4bb0-8629-2e3084a36ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:29.978065 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bb035-4616-4bb0-8629-2e3084a36ccc admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:29.978438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bb035-4616-4bb0-8629-2e3084a36ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:29.978787 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bb035-4616-4bb0-8629-2e3084a36ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:29.979530 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bb035-4616-4bb0-8629-2e3084a36ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:29.981862 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bb035-4616-4bb0-8629-2e3084a36ccc admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:29.996217 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 645/1273] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:29.997157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bb035-4616-4bb0-8629-2e3084a36ccc admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:30.011219 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bb035-4616-4bb0-8629-2e3084a36ccc admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:30.017080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eef33f-2039-4324-9a26-caf4875fd671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:30.017709 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eef33f-2039-4324-9a26-caf4875fd671 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.017990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eef33f-2039-4324-9a26-caf4875fd671 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.018269 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eef33f-2039-4324-9a26-caf4875fd671 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.018917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eef33f-2039-4324-9a26-caf4875fd671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:30.020593 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eef33f-2039-4324-9a26-caf4875fd671 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:30.031008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b70e2a-e8f6-4708-a179-9d98c6475405 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:30.035490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 646/1274] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:30.035490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127626bf-6068-46c3-900e-c5c360811d07 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.035490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127626bf-6068-46c3-900e-c5c360811d07 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.035490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127626bf-6068-46c3-900e-c5c360811d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.039065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7baef2-bcf5-45fe-87dc-bd07e892c77c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:30.039792 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7baef2-bcf5-45fe-87dc-bd07e892c77c service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.040191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7baef2-bcf5-45fe-87dc-bd07e892c77c service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.040557 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7baef2-bcf5-45fe-87dc-bd07e892c77c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.052704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7baef2-bcf5-45fe-87dc-bd07e892c77c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:30.054477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7baef2-bcf5-45fe-87dc-bd07e892c77c service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:30.054824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-127626bf-6068-46c3-900e-c5c360811d07 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:30.056318 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-127626bf-6068-46c3-900e-c5c360811d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:30.065600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 629/1275] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:30.065916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 647/1276] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:30.067121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127626bf-6068-46c3-900e-c5c360811d07 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:30.070512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7baef2-bcf5-45fe-87dc-bd07e892c77c service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:30.079084 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127626bf-6068-46c3-900e-c5c360811d07 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:30.081319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7baef2-bcf5-45fe-87dc-bd07e892c77c service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:30.084543 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32645d27-86e9-4e43-a437-749db84d8e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:30.085377 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32645d27-86e9-4e43-a437-749db84d8e95 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.085714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32645d27-86e9-4e43-a437-749db84d8e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.086121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32645d27-86e9-4e43-a437-749db84d8e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.086774 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32645d27-86e9-4e43-a437-749db84d8e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:30.086945 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9cbaa-30f1-4edb-b1e2-ff27d171e9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:30.087676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9cbaa-30f1-4edb-b1e2-ff27d171e9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.088058 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9cbaa-30f1-4edb-b1e2-ff27d171e9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.088400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9cbaa-30f1-4edb-b1e2-ff27d171e9e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.088952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32645d27-86e9-4e43-a437-749db84d8e95 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:30.089158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9cbaa-30f1-4edb-b1e2-ff27d171e9e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:30.091306 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d9cbaa-30f1-4edb-b1e2-ff27d171e9e4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:30.101979 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 648/1277] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:30.103795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4e040-8bfb-4ef5-b1f3-c4c3674c1d0a None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.104271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4e040-8bfb-4ef5-b1f3-c4c3674c1d0a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.104646 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4e040-8bfb-4ef5-b1f3-c4c3674c1d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.110570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 630/1278] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:30.114678 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9cbaa-30f1-4edb-b1e2-ff27d171e9e4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:30.129363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9cbaa-30f1-4edb-b1e2-ff27d171e9e4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:30.135684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f89bb-7dc7-4307-b9e3-b7bcc2a56455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:30.136457 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f89bb-7dc7-4307-b9e3-b7bcc2a56455 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.136800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f89bb-7dc7-4307-b9e3-b7bcc2a56455 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.137699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f89bb-7dc7-4307-b9e3-b7bcc2a56455 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.138588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f89bb-7dc7-4307-b9e3-b7bcc2a56455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:30.140539 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f89bb-7dc7-4307-b9e3-b7bcc2a56455 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:30.152562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 631/1279] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:30.156651 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f89bb-7dc7-4307-b9e3-b7bcc2a56455 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:30.159758 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-90d4e040-8bfb-4ef5-b1f3-c4c3674c1d0a None None] MFA Rules not processed for user `2e13692c1e83413a909d99bb7db2e4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:30.161292 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-90d4e040-8bfb-4ef5-b1f3-c4c3674c1d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:30.168278 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f89bb-7dc7-4307-b9e3-b7bcc2a56455 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:30.173720 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b6a5d-2cc2-41a7-9ae0-8745593ca01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:30.174454 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b6a5d-2cc2-41a7-9ae0-8745593ca01c admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.174832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b6a5d-2cc2-41a7-9ae0-8745593ca01c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.175184 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b6a5d-2cc2-41a7-9ae0-8745593ca01c admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.176077 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b6a5d-2cc2-41a7-9ae0-8745593ca01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:30.177814 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b6a5d-2cc2-41a7-9ae0-8745593ca01c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:30.192731 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 632/1280] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:30.195304 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b6a5d-2cc2-41a7-9ae0-8745593ca01c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:30.209775 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b6a5d-2cc2-41a7-9ae0-8745593ca01c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:30.217765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52feec2-4c1e-4e12-ba43-770b51084f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:30.218558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52feec2-4c1e-4e12-ba43-770b51084f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.218872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52feec2-4c1e-4e12-ba43-770b51084f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.219173 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52feec2-4c1e-4e12-ba43-770b51084f29 admin admin] PATH_INFO: `/v3/projects/0c29d86a22514388bd9465a4e2fb9daa/users/2c28b2c8bf424d938d875a1af9a7608c/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.241518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52feec2-4c1e-4e12-ba43-770b51084f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c29d86a22514388bd9465a4e2fb9daa, user_id=2c28b2c8bf424d938d875a1af9a7608c, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:30.245245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52feec2-4c1e-4e12-ba43-770b51084f29 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:30.263660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 633/1281] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:30 2022] PUT /identity/v3/projects/0c29d86a22514388bd9465a4e2fb9daa/users/2c28b2c8bf424d938d875a1af9a7608c/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:30.265122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52feec2-4c1e-4e12-ba43-770b51084f29 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:30.277661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 649/1282] 10.222.0.36 () {58 vars in 1058 bytes} [Sat Oct 15 13:48:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:30.279138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4e040-8bfb-4ef5-b1f3-c4c3674c1d0a None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:30.294770 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4e040-8bfb-4ef5-b1f3-c4c3674c1d0a None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:30.357575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52feec2-4c1e-4e12-ba43-770b51084f29 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:30.365293 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca7e1f-4d48-4063-982d-a503461bc058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:30.366391 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca7e1f-4d48-4063-982d-a503461bc058 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.366986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca7e1f-4d48-4063-982d-a503461bc058 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.367763 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca7e1f-4d48-4063-982d-a503461bc058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.375356 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:30.393359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a121a1-149c-4bac-873c-97b9276e72ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:30.394521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a121a1-149c-4bac-873c-97b9276e72ce admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.395074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a121a1-149c-4bac-873c-97b9276e72ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.396036 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a121a1-149c-4bac-873c-97b9276e72ce admin admin] PATH_INFO: `/v3/projects/2f75462e70f94e8a8ee4d907be539416/users/667cdddd4e314f60b8b2a6ddc64152df/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.423347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a121a1-149c-4bac-873c-97b9276e72ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f75462e70f94e8a8ee4d907be539416, user_id=667cdddd4e314f60b8b2a6ddc64152df, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:30.428691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a121a1-149c-4bac-873c-97b9276e72ce admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:30.452533 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 650/1283] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:30 2022] PUT /identity/v3/projects/2f75462e70f94e8a8ee4d907be539416/users/667cdddd4e314f60b8b2a6ddc64152df/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:30.454241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a121a1-149c-4bac-873c-97b9276e72ce admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:30.483557 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca7e1f-4d48-4063-982d-a503461bc058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:30.485368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca7e1f-4d48-4063-982d-a503461bc058 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:30.496753 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 634/1284] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:30.508044 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca7e1f-4d48-4063-982d-a503461bc058 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:30.516810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:30.520590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca7e1f-4d48-4063-982d-a503461bc058 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:30.592885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a121a1-149c-4bac-873c-97b9276e72ce admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:30.602388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee56e12-a431-4381-9aaf-a7a4c299ac7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:30.606779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee56e12-a431-4381-9aaf-a7a4c299ac7a admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.606779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee56e12-a431-4381-9aaf-a7a4c299ac7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.607531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee56e12-a431-4381-9aaf-a7a4c299ac7a admin admin] PATH_INFO: `/v3/projects/735c94e76c1e4aca8c1b82ea5d744e62/users/4d5c3d2aebd94d34a335d685343fe129/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.640670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4538cd-e779-43bc-b6e0-95400129d8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:30.642863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee56e12-a431-4381-9aaf-a7a4c299ac7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=735c94e76c1e4aca8c1b82ea5d744e62, user_id=4d5c3d2aebd94d34a335d685343fe129, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:30.643772 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4538cd-e779-43bc-b6e0-95400129d8c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.644377 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4538cd-e779-43bc-b6e0-95400129d8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.645227 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4538cd-e779-43bc-b6e0-95400129d8c2 admin admin] PATH_INFO: `/v3/projects/09a0813bae074cd2aa552be82abe8493/users/5150c06020594efe957359f641eba5da/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.649665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee56e12-a431-4381-9aaf-a7a4c299ac7a admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:30.693305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4538cd-e779-43bc-b6e0-95400129d8c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09a0813bae074cd2aa552be82abe8493, user_id=5150c06020594efe957359f641eba5da, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:30.706402 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 651/1285] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:30 2022] PUT /identity/v3/projects/735c94e76c1e4aca8c1b82ea5d744e62/users/4d5c3d2aebd94d34a335d685343fe129/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:30.706402 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee56e12-a431-4381-9aaf-a7a4c299ac7a admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:30.707605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4538cd-e779-43bc-b6e0-95400129d8c2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:30.736416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 635/1286] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:30 2022] PUT /identity/v3/projects/09a0813bae074cd2aa552be82abe8493/users/5150c06020594efe957359f641eba5da/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:30.738879 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbed52b-d34d-4986-8ec9-ae6cba775971 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.739373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbed52b-d34d-4986-8ec9-ae6cba775971 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.739854 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbed52b-d34d-4986-8ec9-ae6cba775971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.794176 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ddbed52b-d34d-4986-8ec9-ae6cba775971 None None] MFA Rules not processed for user `2c28b2c8bf424d938d875a1af9a7608c`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:30.795951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ddbed52b-d34d-4986-8ec9-ae6cba775971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:30.803233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee56e12-a431-4381-9aaf-a7a4c299ac7a admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:30.941639 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db84ca-10fc-44e4-bfdb-edc8620cdf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:30.942338 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db84ca-10fc-44e4-bfdb-edc8620cdf61 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:30.942614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db84ca-10fc-44e4-bfdb-edc8620cdf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:30.942884 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db84ca-10fc-44e4-bfdb-edc8620cdf61 admin admin] PATH_INFO: `/v3/projects/aabf1193b43040948317be442c5c16f7/users/0b01e277944d4edcb8625ebf4544e340/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:30.952762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 636/1287] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:30.955344 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbed52b-d34d-4986-8ec9-ae6cba775971 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:30.957711 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:30.971098 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db84ca-10fc-44e4-bfdb-edc8620cdf61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aabf1193b43040948317be442c5c16f7, user_id=0b01e277944d4edcb8625ebf4544e340, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:30.975915 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db84ca-10fc-44e4-bfdb-edc8620cdf61 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:31.001702 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 652/1288] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:30 2022] PUT /identity/v3/projects/aabf1193b43040948317be442c5c16f7/users/0b01e277944d4edcb8625ebf4544e340/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:31.006186 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d73b7-acb0-46ad-8a54-080091fd4b46 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:31.006186 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d73b7-acb0-46ad-8a54-080091fd4b46 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:31.006186 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d73b7-acb0-46ad-8a54-080091fd4b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:31.026674 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-5f9d73b7-acb0-46ad-8a54-080091fd4b46 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:31.028135 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-5f9d73b7-acb0-46ad-8a54-080091fd4b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:31.132050 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 653/1289] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:31.132050 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64c68b-f149-4742-b547-b75b1f2df8f6 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:31.132050 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64c68b-f149-4742-b547-b75b1f2df8f6 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:31.132482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64c68b-f149-4742-b547-b75b1f2df8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:31.183244 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbed52b-d34d-4986-8ec9-ae6cba775971 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:31.188108 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-fd64c68b-f149-4742-b547-b75b1f2df8f6 None None] MFA Rules not processed for user `4d5c3d2aebd94d34a335d685343fe129`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:31.189389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-fd64c68b-f149-4742-b547-b75b1f2df8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:31.208772 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:31.305296 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 654/1290] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:31.308109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da379d0-ef4c-4505-8b31-7899c65cd027 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:31.308616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da379d0-ef4c-4505-8b31-7899c65cd027 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:31.309001 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da379d0-ef4c-4505-8b31-7899c65cd027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:31.337272 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-6da379d0-ef4c-4505-8b31-7899c65cd027 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:31.338917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6da379d0-ef4c-4505-8b31-7899c65cd027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:31.353458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 655/1291] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:31.353458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88980fd-a925-44cd-9d2f-9dddc4dbc6a9 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:31.353458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88980fd-a925-44cd-9d2f-9dddc4dbc6a9 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:31.353458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88980fd-a925-44cd-9d2f-9dddc4dbc6a9 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:31.354501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 656/1292] 10.222.0.36 () {54 vars in 961 bytes} [Sat Oct 15 13:48:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:48:31.355509 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e185f44-2e23-4fb6-bbc6-a50a660a1d98 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:31.355794 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e185f44-2e23-4fb6-bbc6-a50a660a1d98 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:31.356091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e185f44-2e23-4fb6-bbc6-a50a660a1d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:31.358676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993897d-3182-48e0-b8fc-10df3694f7ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:31.361975 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993897d-3182-48e0-b8fc-10df3694f7ee service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:31.362352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993897d-3182-48e0-b8fc-10df3694f7ee service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:31.364101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993897d-3182-48e0-b8fc-10df3694f7ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:31.382101 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-9e185f44-2e23-4fb6-bbc6-a50a660a1d98 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:31.383962 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9e185f44-2e23-4fb6-bbc6-a50a660a1d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:31.395628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 657/1293] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:31.395628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e185f44-2e23-4fb6-bbc6-a50a660a1d98 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:31.498894 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993897d-3182-48e0-b8fc-10df3694f7ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:31.503156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993897d-3182-48e0-b8fc-10df3694f7ee service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:31.514442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e185f44-2e23-4fb6-bbc6-a50a660a1d98 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:31.523401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 637/1294] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:31.525847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328c601-2dcc-4473-bb47-76d3d85376e8 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:31.526449 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328c601-2dcc-4473-bb47-76d3d85376e8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:31.526790 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328c601-2dcc-4473-bb47-76d3d85376e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:31.537070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c3194-03cb-43b8-924a-28b61540ac55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:31.538047 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c3194-03cb-43b8-924a-28b61540ac55 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:31.539861 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c3194-03cb-43b8-924a-28b61540ac55 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:31.540355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c3194-03cb-43b8-924a-28b61540ac55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:31.557445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6328c601-2dcc-4473-bb47-76d3d85376e8 None None] MFA Rules not processed for user `2e13692c1e83413a909d99bb7db2e4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:31.559626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6328c601-2dcc-4473-bb47-76d3d85376e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:31.576531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c3194-03cb-43b8-924a-28b61540ac55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:31.579042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c3194-03cb-43b8-924a-28b61540ac55 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:31.594062 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 638/1295] 10.222.0.36 () {58 vars in 1058 bytes} [Sat Oct 15 13:48:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:31.595104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328c601-2dcc-4473-bb47-76d3d85376e8 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:31.599520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 658/1296] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:31.600404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c3194-03cb-43b8-924a-28b61540ac55 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:31.620099 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c3194-03cb-43b8-924a-28b61540ac55 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:31.629719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328c601-2dcc-4473-bb47-76d3d85376e8 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:31.636029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7219a-aee1-4847-9b36-51afc41766b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:31.637137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7219a-aee1-4847-9b36-51afc41766b2 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:31.637645 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7219a-aee1-4847-9b36-51afc41766b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:31.638431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7219a-aee1-4847-9b36-51afc41766b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:31.639573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c92805-3e5b-4db6-9d9b-387d4446964b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:31.640695 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c92805-3e5b-4db6-9d9b-387d4446964b service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:31.641195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c92805-3e5b-4db6-9d9b-387d4446964b service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:31.641737 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c92805-3e5b-4db6-9d9b-387d4446964b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:31.652705 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:31.662710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c92805-3e5b-4db6-9d9b-387d4446964b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:31.666189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c92805-3e5b-4db6-9d9b-387d4446964b service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:31.667233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7219a-aee1-4847-9b36-51afc41766b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:31.668938 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7219a-aee1-4847-9b36-51afc41766b2 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:31.689067 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 659/1297] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:31.690025 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7219a-aee1-4847-9b36-51afc41766b2 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:31.716790 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 639/1298] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:31.720680 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7219a-aee1-4847-9b36-51afc41766b2 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:31.730369 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632e9f8-bf0e-4f11-a2bf-c8856801f5a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:31.731015 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632e9f8-bf0e-4f11-a2bf-c8856801f5a0 service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:31.731288 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632e9f8-bf0e-4f11-a2bf-c8856801f5a0 service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:31.731562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632e9f8-bf0e-4f11-a2bf-c8856801f5a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:31.900810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c92805-3e5b-4db6-9d9b-387d4446964b service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:31.914463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632e9f8-bf0e-4f11-a2bf-c8856801f5a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:31.916818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632e9f8-bf0e-4f11-a2bf-c8856801f5a0 service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:31.931076 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 660/1299] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:31.939025 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c92805-3e5b-4db6-9d9b-387d4446964b service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:31.945722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ec278-b3ad-4ef4-89b0-f6152cc5bc4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:31.946449 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ec278-b3ad-4ef4-89b0-f6152cc5bc4e service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:31.946810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ec278-b3ad-4ef4-89b0-f6152cc5bc4e service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:31.947144 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ec278-b3ad-4ef4-89b0-f6152cc5bc4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:31.971101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ec278-b3ad-4ef4-89b0-f6152cc5bc4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:31.979755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ec278-b3ad-4ef4-89b0-f6152cc5bc4e service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:32.034868 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 640/1300] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:48:32.075939 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb88661-787f-4963-bc57-2bd74104039f None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:32.076466 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb88661-787f-4963-bc57-2bd74104039f None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:32.076466 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb88661-787f-4963-bc57-2bd74104039f None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:32.078477 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 661/1301] 10.222.0.36 () {54 vars in 912 bytes} [Sat Oct 15 13:48:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:48:32.088027 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab01f2-4a45-4267-abb6-e4fed0aff526 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:32.089402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab01f2-4a45-4267-abb6-e4fed0aff526 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:32.090547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab01f2-4a45-4267-abb6-e4fed0aff526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:32.179641 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ebab01f2-4a45-4267-abb6-e4fed0aff526 None None] MFA Rules not processed for user `2e13692c1e83413a909d99bb7db2e4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:32.184569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ebab01f2-4a45-4267-abb6-e4fed0aff526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:32.212602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 641/1302] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:48:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:32.227511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb88661-787f-4963-bc57-2bd74104039f None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:32.365399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b9cb8-b378-4e3f-86a2-0d9320afa6ea None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:32.365695 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b9cb8-b378-4e3f-86a2-0d9320afa6ea None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:32.365972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b9cb8-b378-4e3f-86a2-0d9320afa6ea None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:32.368978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 642/1303] 10.222.0.36 () {54 vars in 912 bytes} [Sat Oct 15 13:48:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:48:32.371329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb88661-787f-4963-bc57-2bd74104039f None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:32.377615 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636b664-1f76-4fb4-b8c7-e198d94771bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:32.378779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636b664-1f76-4fb4-b8c7-e198d94771bd service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:32.379212 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636b664-1f76-4fb4-b8c7-e198d94771bd service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:32.379643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636b664-1f76-4fb4-b8c7-e198d94771bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:32.390045 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efb42d-5615-47cc-9923-a69daa3ce764 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:32.390310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efb42d-5615-47cc-9923-a69daa3ce764 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:32.390583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efb42d-5615-47cc-9923-a69daa3ce764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:32.412163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636b664-1f76-4fb4-b8c7-e198d94771bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:32.415749 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636b664-1f76-4fb4-b8c7-e198d94771bd service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:32.431863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a9efb42d-5615-47cc-9923-a69daa3ce764 None None] MFA Rules not processed for user `2e13692c1e83413a909d99bb7db2e4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:32.433960 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a9efb42d-5615-47cc-9923-a69daa3ce764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:32.454518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 643/1304] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:48:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:48:32.467347 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efb42d-5615-47cc-9923-a69daa3ce764 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:32.469920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 662/1305] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:32.485070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efb42d-5615-47cc-9923-a69daa3ce764 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:32.494604 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ce297-1f1a-4770-93a8-0cdd378cf057 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:32.496110 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ce297-1f1a-4770-93a8-0cdd378cf057 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:32.496831 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ce297-1f1a-4770-93a8-0cdd378cf057 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:32.498744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ce297-1f1a-4770-93a8-0cdd378cf057 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:32.522888 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ce297-1f1a-4770-93a8-0cdd378cf057 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:32.527121 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ce297-1f1a-4770-93a8-0cdd378cf057 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:32.560664 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 644/1306] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:48:34.159196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636b664-1f76-4fb4-b8c7-e198d94771bd service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:34.183004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636b664-1f76-4fb4-b8c7-e198d94771bd service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:34.193515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:34.194269 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:34.194647 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:34.194994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] PATH_INFO: `/v3/users/8b3f059dc7e04446a8f04da22761c30f` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:34.197214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b3f059dc7e04446a8f04da22761c30f)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:34.199636 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: WARNING py.warnings [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:48:34.199636 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: warnings.warn(msg) Oct 15 13:48:34.199636 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: Oct 15 13:48:34.202400 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:34.406658 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b3f059dc7e04446a8f04da22761c30f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:34.427727 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b3f059dc7e04446a8f04da22761c30f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:34.428687 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b3f059dc7e04446a8f04da22761c30f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:34.429682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b3f059dc7e04446a8f04da22761c30f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:34.449565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b3f059dc7e04446a8f04da22761c30f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:34.539949 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 663/1307] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:48:34 2022] DELETE /identity/v3/users/8b3f059dc7e04446a8f04da22761c30f => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:34.546307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417412d-8d45-4d56-a53e-22a7e7dc4986 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:34.547236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417412d-8d45-4d56-a53e-22a7e7dc4986 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:34.547236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417412d-8d45-4d56-a53e-22a7e7dc4986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:34.584515 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8417412d-8d45-4d56-a53e-22a7e7dc4986 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:34.585782 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8417412d-8d45-4d56-a53e-22a7e7dc4986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:34.680545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:34.727434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 645/1308] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:48:34.774902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:34.777601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:34.892101 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:34.998651 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f2833-6aa0-4295-9f2e-1aca31a01264 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:35.003617 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938efab-9e2d-48d0-9a11-dd5d8b59951c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:35.004433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938efab-9e2d-48d0-9a11-dd5d8b59951c service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:35.004821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938efab-9e2d-48d0-9a11-dd5d8b59951c service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:35.005152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938efab-9e2d-48d0-9a11-dd5d8b59951c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:35.024374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f938efab-9e2d-48d0-9a11-dd5d8b59951c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:35.026035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f938efab-9e2d-48d0-9a11-dd5d8b59951c service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:35.045575 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 664/1309] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:35.492358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ecbff-d3e8-45ae-aa8b-7fb5efc87619 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:35.492702 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ecbff-d3e8-45ae-aa8b-7fb5efc87619 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:35.492859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ecbff-d3e8-45ae-aa8b-7fb5efc87619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:35.569275 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7a2ecbff-d3e8-45ae-aa8b-7fb5efc87619 None None] MFA Rules not processed for user `667cdddd4e314f60b8b2a6ddc64152df`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:35.570911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7a2ecbff-d3e8-45ae-aa8b-7fb5efc87619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:36.006232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 646/1310] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:35 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 515 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:48:36.012155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938efab-9e2d-48d0-9a11-dd5d8b59951c service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:36.016119 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f938efab-9e2d-48d0-9a11-dd5d8b59951c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:36.042652 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938efab-9e2d-48d0-9a11-dd5d8b59951c service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:36.069293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e8a35-089d-4fac-a274-a125018406c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:36.070214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e8a35-089d-4fac-a274-a125018406c4 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:36.070694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e8a35-089d-4fac-a274-a125018406c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:36.073147 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e8a35-089d-4fac-a274-a125018406c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:36.074068 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e8a35-089d-4fac-a274-a125018406c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:36.082141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e8a35-089d-4fac-a274-a125018406c4 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:36.167535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 665/1311] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:36 2022] POST /identity/v3/projects => generated 403 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:36.173586 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ecbff-d3e8-45ae-aa8b-7fb5efc87619 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:36.257588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ecbff-d3e8-45ae-aa8b-7fb5efc87619 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:36.552606 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dcc08-4f7a-4842-8256-5110333f7c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:36.553764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dcc08-4f7a-4842-8256-5110333f7c01 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:36.554248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dcc08-4f7a-4842-8256-5110333f7c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:36.554707 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dcc08-4f7a-4842-8256-5110333f7c01 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:36.561385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dcc08-4f7a-4842-8256-5110333f7c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:36.567987 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dcc08-4f7a-4842-8256-5110333f7c01 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:36.648481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 647/1312] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:36 2022] POST /identity/v3/users => generated 308 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:36.654496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e8a35-089d-4fac-a274-a125018406c4 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:36.692238 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e8a35-089d-4fac-a274-a125018406c4 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:36.701276 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97a871-bda7-41e3-894f-c3d7c1b79b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:36.702184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97a871-bda7-41e3-894f-c3d7c1b79b39 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:36.702526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97a871-bda7-41e3-894f-c3d7c1b79b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:36.702797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97a871-bda7-41e3-894f-c3d7c1b79b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:36.703952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97a871-bda7-41e3-894f-c3d7c1b79b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:36.706503 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97a871-bda7-41e3-894f-c3d7c1b79b39 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:36.724288 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 666/1313] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:48:36.741471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dcc08-4f7a-4842-8256-5110333f7c01 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:36.755088 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dcc08-4f7a-4842-8256-5110333f7c01 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:36.782569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da48da-861e-4d2b-9e48-92c3f5517e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:36.787375 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da48da-861e-4d2b-9e48-92c3f5517e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:36.787558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da48da-861e-4d2b-9e48-92c3f5517e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:36.787731 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da48da-861e-4d2b-9e48-92c3f5517e43 admin admin] PATH_INFO: `/v3/projects/73abe7085bd341f3bca2322efa9f16a7/users/824a4131151746288637abade5ab9dfd/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:36.836047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da48da-861e-4d2b-9e48-92c3f5517e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73abe7085bd341f3bca2322efa9f16a7, user_id=824a4131151746288637abade5ab9dfd, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:36.844359 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da48da-861e-4d2b-9e48-92c3f5517e43 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:36.884167 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 648/1314] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:36 2022] PUT /identity/v3/projects/73abe7085bd341f3bca2322efa9f16a7/users/824a4131151746288637abade5ab9dfd/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:36.890140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97a871-bda7-41e3-894f-c3d7c1b79b39 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:36.911451 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97a871-bda7-41e3-894f-c3d7c1b79b39 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:37.086399 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611ea1b-474c-444f-b2ad-ebe44fa84d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:37.088079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611ea1b-474c-444f-b2ad-ebe44fa84d6e admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:37.089060 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611ea1b-474c-444f-b2ad-ebe44fa84d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:37.089616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611ea1b-474c-444f-b2ad-ebe44fa84d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:37.090634 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611ea1b-474c-444f-b2ad-ebe44fa84d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:37.093379 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611ea1b-474c-444f-b2ad-ebe44fa84d6e admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:37.110658 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 667/1315] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:37.116727 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da48da-861e-4d2b-9e48-92c3f5517e43 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:37.133521 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da48da-861e-4d2b-9e48-92c3f5517e43 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:37.148937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c6bdd-1e13-496e-8f82-bd56fc906ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:37.150008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c6bdd-1e13-496e-8f82-bd56fc906ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:37.150490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c6bdd-1e13-496e-8f82-bd56fc906ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:37.150844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c6bdd-1e13-496e-8f82-bd56fc906ae4 admin admin] PATH_INFO: `/v3/projects/73abe7085bd341f3bca2322efa9f16a7/users/824a4131151746288637abade5ab9dfd/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:37.153433 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c6bdd-1e13-496e-8f82-bd56fc906ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73abe7085bd341f3bca2322efa9f16a7, user_id=824a4131151746288637abade5ab9dfd, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:37.160754 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c6bdd-1e13-496e-8f82-bd56fc906ae4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:37.201380 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 649/1316] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:37 2022] PUT /identity/v3/projects/73abe7085bd341f3bca2322efa9f16a7/users/824a4131151746288637abade5ab9dfd/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:48:38.693906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611ea1b-474c-444f-b2ad-ebe44fa84d6e admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:38.696928 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3611ea1b-474c-444f-b2ad-ebe44fa84d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:38.745383 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611ea1b-474c-444f-b2ad-ebe44fa84d6e admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:38.910214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe6d12-7cf1-48d5-a491-61047981ece3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:38.911290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe6d12-7cf1-48d5-a491-61047981ece3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:38.911762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe6d12-7cf1-48d5-a491-61047981ece3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:38.912655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe6d12-7cf1-48d5-a491-61047981ece3 admin admin] PATH_INFO: `/v3/projects/080f5405f5a44718858ccbc99dc49680` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:38.916274 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe6d12-7cf1-48d5-a491-61047981ece3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=080f5405f5a44718858ccbc99dc49680)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:38.922488 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe6d12-7cf1-48d5-a491-61047981ece3 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:38.963256 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:39.132418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9bbe6d12-7cf1-48d5-a491-61047981ece3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '080f5405f5a44718858ccbc99dc49680', 'request_id': 'req-9bbe6d12-7cf1-48d5-a491-61047981ece3'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:39.164187 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9bbe6d12-7cf1-48d5-a491-61047981ece3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '080f5405f5a44718858ccbc99dc49680', 'request_id': 'req-9bbe6d12-7cf1-48d5-a491-61047981ece3'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:39.187478 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9bbe6d12-7cf1-48d5-a491-61047981ece3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '080f5405f5a44718858ccbc99dc49680', 'request_id': 'req-9bbe6d12-7cf1-48d5-a491-61047981ece3'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:39.241261 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9bbe6d12-7cf1-48d5-a491-61047981ece3 admin admin] The token cache is being invalidate because project 080f5405f5a44718858ccbc99dc49680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:48:39.241261 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9bbe6d12-7cf1-48d5-a491-61047981ece3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:39.243784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9bbe6d12-7cf1-48d5-a491-61047981ece3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:39.246549 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 668/1317] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:48:38 2022] DELETE /identity/v3/projects/080f5405f5a44718858ccbc99dc49680 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:39.269732 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ce133-0a1f-4ed2-97e3-9af5e3c8a7ca None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:39.271125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ce133-0a1f-4ed2-97e3-9af5e3c8a7ca None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:39.271809 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ce133-0a1f-4ed2-97e3-9af5e3c8a7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:39.423262 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-558ce133-0a1f-4ed2-97e3-9af5e3c8a7ca None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:39.429197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-558ce133-0a1f-4ed2-97e3-9af5e3c8a7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:39.631036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:39.820852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 650/1318] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 567 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:39.835252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3f236-c64d-4619-bc05-d26baa83fb68 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:39.836362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3f236-c64d-4619-bc05-d26baa83fb68 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:39.836969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3f236-c64d-4619-bc05-d26baa83fb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:39.901168 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-65f3f236-c64d-4619-bc05-d26baa83fb68 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:39.902469 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-65f3f236-c64d-4619-bc05-d26baa83fb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:39.942933 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 669/1319] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:39.944660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ce133-0a1f-4ed2-97e3-9af5e3c8a7ca None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:40.012527 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ce133-0a1f-4ed2-97e3-9af5e3c8a7ca None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:40.055583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d2e9f-a83d-4e90-b008-30c44efdc861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:40.056955 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d2e9f-a83d-4e90-b008-30c44efdc861 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:40.057435 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d2e9f-a83d-4e90-b008-30c44efdc861 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:40.057912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d2e9f-a83d-4e90-b008-30c44efdc861 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:40.059117 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d2e9f-a83d-4e90-b008-30c44efdc861 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:48:40.059736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d2e9f-a83d-4e90-b008-30c44efdc861 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:40.063178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d2e9f-a83d-4e90-b008-30c44efdc861 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:40.141543 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3f236-c64d-4619-bc05-d26baa83fb68 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:40.143585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 651/1320] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:48:40.158965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3f236-c64d-4619-bc05-d26baa83fb68 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:40.173010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe754d5-2389-4421-bb15-6848bc11e9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:40.174501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe754d5-2389-4421-bb15-6848bc11e9e4 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:40.175022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe754d5-2389-4421-bb15-6848bc11e9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:40.175514 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe754d5-2389-4421-bb15-6848bc11e9e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:40.176697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe754d5-2389-4421-bb15-6848bc11e9e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:40.180011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe754d5-2389-4421-bb15-6848bc11e9e4 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:40.258430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 670/1321] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:40 2022] POST /identity/v3/projects => generated 363 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:40.265639 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d2e9f-a83d-4e90-b008-30c44efdc861 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:40.301807 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d2e9f-a83d-4e90-b008-30c44efdc861 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:40.680401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e6127-b2f7-41fe-8d9b-4dec44291056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:40.681830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e6127-b2f7-41fe-8d9b-4dec44291056 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:40.682316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e6127-b2f7-41fe-8d9b-4dec44291056 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:40.683334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e6127-b2f7-41fe-8d9b-4dec44291056 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:40.684622 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e6127-b2f7-41fe-8d9b-4dec44291056 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:40.691532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e6127-b2f7-41fe-8d9b-4dec44291056 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:40.812268 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 652/1322] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:40 2022] POST /identity/v3/users => generated 282 bytes in 548 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:40.813355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe754d5-2389-4421-bb15-6848bc11e9e4 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:40.834738 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe754d5-2389-4421-bb15-6848bc11e9e4 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:40.851714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ade19-5040-4d44-92a9-adf61bdda270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:40.853956 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ade19-5040-4d44-92a9-adf61bdda270 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:40.854279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ade19-5040-4d44-92a9-adf61bdda270 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:40.854548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ade19-5040-4d44-92a9-adf61bdda270 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:40.856428 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ade19-5040-4d44-92a9-adf61bdda270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:40.863098 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ade19-5040-4d44-92a9-adf61bdda270 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:40.904404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 671/1323] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:40.911272 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e6127-b2f7-41fe-8d9b-4dec44291056 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:40.939304 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e6127-b2f7-41fe-8d9b-4dec44291056 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:40.963835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94f525-9120-4d68-8e30-e970b28839de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:40.965011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94f525-9120-4d68-8e30-e970b28839de admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:40.965575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94f525-9120-4d68-8e30-e970b28839de admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:40.966383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94f525-9120-4d68-8e30-e970b28839de admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:40.971036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94f525-9120-4d68-8e30-e970b28839de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:40.975361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94f525-9120-4d68-8e30-e970b28839de admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:41.039005 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ade19-5040-4d44-92a9-adf61bdda270 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:41.053619 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 653/1324] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:41.071985 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ade19-5040-4d44-92a9-adf61bdda270 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:41.088274 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f67d-a5e8-485c-827d-e1fba18e0b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:41.090030 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f67d-a5e8-485c-827d-e1fba18e0b54 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:41.091103 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f67d-a5e8-485c-827d-e1fba18e0b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:41.092295 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f67d-a5e8-485c-827d-e1fba18e0b54 admin admin] PATH_INFO: `/v3/projects/f8b834400955461d9dbe38f92950eb1e/users/1a23a7ac7dbb469b87a326337d63ae7c/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:41.149898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4f67d-a5e8-485c-827d-e1fba18e0b54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b834400955461d9dbe38f92950eb1e, user_id=1a23a7ac7dbb469b87a326337d63ae7c, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:41.158883 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4f67d-a5e8-485c-827d-e1fba18e0b54 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:41.232817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 672/1325] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:41 2022] PUT /identity/v3/projects/f8b834400955461d9dbe38f92950eb1e/users/1a23a7ac7dbb469b87a326337d63ae7c/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:41.247348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d821cd-4811-48fb-a969-deaa0c86fbb6 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:41.247800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d821cd-4811-48fb-a969-deaa0c86fbb6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:41.248387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d821cd-4811-48fb-a969-deaa0c86fbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:41.374492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e9d821cd-4811-48fb-a969-deaa0c86fbb6 None None] MFA Rules not processed for user `1a23a7ac7dbb469b87a326337d63ae7c`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:41.377019 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e9d821cd-4811-48fb-a969-deaa0c86fbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:41.641799 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f67d-a5e8-485c-827d-e1fba18e0b54 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:41.643482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 654/1326] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:41.913165 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f67d-a5e8-485c-827d-e1fba18e0b54 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:41.933990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d18ef0-cdaf-4296-bd6e-d54c42f3c998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:41.935118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d18ef0-cdaf-4296-bd6e-d54c42f3c998 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:41.935599 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d18ef0-cdaf-4296-bd6e-d54c42f3c998 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:41.936119 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d18ef0-cdaf-4296-bd6e-d54c42f3c998 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:41.945828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d18ef0-cdaf-4296-bd6e-d54c42f3c998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:41.950010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d18ef0-cdaf-4296-bd6e-d54c42f3c998 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:42.056946 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 673/1327] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:41 2022] POST /identity/v3/projects => generated 365 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:42.058371 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d821cd-4811-48fb-a969-deaa0c86fbb6 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:42.071366 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d821cd-4811-48fb-a969-deaa0c86fbb6 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:42.145397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:42.218027 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b963c59-4bd5-48fc-b726-7cd4030f86cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:42.218706 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b963c59-4bd5-48fc-b726-7cd4030f86cd admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:42.223253 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b963c59-4bd5-48fc-b726-7cd4030f86cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:42.223569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b963c59-4bd5-48fc-b726-7cd4030f86cd admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:42.224315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b963c59-4bd5-48fc-b726-7cd4030f86cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:42.231847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b963c59-4bd5-48fc-b726-7cd4030f86cd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:42.328367 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 655/1328] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:42 2022] POST /identity/v3/users => generated 289 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:42.330311 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d18ef0-cdaf-4296-bd6e-d54c42f3c998 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:42.396950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d18ef0-cdaf-4296-bd6e-d54c42f3c998 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:42.417444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cd4ed-bf2f-4806-8dc7-14fbf61e7ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:42.418228 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cd4ed-bf2f-4806-8dc7-14fbf61e7ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:42.418512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cd4ed-bf2f-4806-8dc7-14fbf61e7ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:42.418786 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cd4ed-bf2f-4806-8dc7-14fbf61e7ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:42.420431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cd4ed-bf2f-4806-8dc7-14fbf61e7ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:42.425291 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cd4ed-bf2f-4806-8dc7-14fbf61e7ae7 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:42.456453 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 674/1329] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:42.461566 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b963c59-4bd5-48fc-b726-7cd4030f86cd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:42.494826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b963c59-4bd5-48fc-b726-7cd4030f86cd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:42.514030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075751f3-2477-4bf7-b917-4d99cb5278c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:42.515261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075751f3-2477-4bf7-b917-4d99cb5278c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:42.519560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075751f3-2477-4bf7-b917-4d99cb5278c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:42.520681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075751f3-2477-4bf7-b917-4d99cb5278c8 admin admin] PATH_INFO: `/v3/projects/a077efef66d24d7aa9f5f71d3d9c7360/users/7f3cb71962c84777bff5f8c651d85300/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:42.569173 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075751f3-2477-4bf7-b917-4d99cb5278c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a077efef66d24d7aa9f5f71d3d9c7360, user_id=7f3cb71962c84777bff5f8c651d85300, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:42.572615 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075751f3-2477-4bf7-b917-4d99cb5278c8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:42.604564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 656/1330] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:42 2022] PUT /identity/v3/projects/a077efef66d24d7aa9f5f71d3d9c7360/users/7f3cb71962c84777bff5f8c651d85300/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:42.607983 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cd4ed-bf2f-4806-8dc7-14fbf61e7ae7 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:42.647236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac4c42-ae4f-40ad-9cf3-8c040c529660 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:42.647894 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac4c42-ae4f-40ad-9cf3-8c040c529660 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:42.648322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac4c42-ae4f-40ad-9cf3-8c040c529660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:42.651758 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cd4ed-bf2f-4806-8dc7-14fbf61e7ae7 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:42.673328 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:42.749798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a9ac4c42-ae4f-40ad-9cf3-8c040c529660 None None] MFA Rules not processed for user `824a4131151746288637abade5ab9dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:42.751085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a9ac4c42-ae4f-40ad-9cf3-8c040c529660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:42.912891 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebe672-2e2c-4ac4-89ce-d4259af06f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:42.921070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebe672-2e2c-4ac4-89ce-d4259af06f49 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:42.921779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebe672-2e2c-4ac4-89ce-d4259af06f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:42.924267 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebe672-2e2c-4ac4-89ce-d4259af06f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:42.925686 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ebe672-2e2c-4ac4-89ce-d4259af06f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:42.942199 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ebe672-2e2c-4ac4-89ce-d4259af06f49 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:42.993920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 675/1331] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:43.007971 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebe672-2e2c-4ac4-89ce-d4259af06f49 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:43.052840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebe672-2e2c-4ac4-89ce-d4259af06f49 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:43.078781 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c61730-3cbd-45a7-b1d5-422a0c8ab52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:43.080341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c61730-3cbd-45a7-b1d5-422a0c8ab52b admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:43.081797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c61730-3cbd-45a7-b1d5-422a0c8ab52b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:43.083412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c61730-3cbd-45a7-b1d5-422a0c8ab52b admin admin] PATH_INFO: `/v3/projects/a077efef66d24d7aa9f5f71d3d9c7360/users/7f3cb71962c84777bff5f8c651d85300/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:43.090426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c61730-3cbd-45a7-b1d5-422a0c8ab52b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a077efef66d24d7aa9f5f71d3d9c7360, user_id=7f3cb71962c84777bff5f8c651d85300, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:43.103561 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 657/1332] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:42 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:48:43.115481 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c61730-3cbd-45a7-b1d5-422a0c8ab52b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:43.163769 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 676/1333] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:43 2022] PUT /identity/v3/projects/a077efef66d24d7aa9f5f71d3d9c7360/users/7f3cb71962c84777bff5f8c651d85300/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:43.177345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c4b2-ccf7-4985-b730-9015b5f03fe6 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:43.177935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c4b2-ccf7-4985-b730-9015b5f03fe6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:43.178478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c4b2-ccf7-4985-b730-9015b5f03fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:43.295056 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-dbe5c4b2-ccf7-4985-b730-9015b5f03fe6 None None] MFA Rules not processed for user `7f3cb71962c84777bff5f8c651d85300`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:43.298225 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dbe5c4b2-ccf7-4985-b730-9015b5f03fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:43.655199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 658/1334] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:43.664112 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c61730-3cbd-45a7-b1d5-422a0c8ab52b admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:43.667761 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-d0c61730-3cbd-45a7-b1d5-422a0c8ab52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:43.671162 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5c4b2-ccf7-4985-b730-9015b5f03fe6 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:43.673657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dbe5c4b2-ccf7-4985-b730-9015b5f03fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:43.706753 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:43.971282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c61730-3cbd-45a7-b1d5-422a0c8ab52b admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:43.989960 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5c4b2-ccf7-4985-b730-9015b5f03fe6 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:44.005191 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c19be-99a6-4550-abc8-5b1b13a510ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:44.006712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c19be-99a6-4550-abc8-5b1b13a510ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:44.007335 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c19be-99a6-4550-abc8-5b1b13a510ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:44.008828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c19be-99a6-4550-abc8-5b1b13a510ae admin admin] PATH_INFO: `/v3/users/2c28b2c8bf424d938d875a1af9a7608c` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:44.017510 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c19be-99a6-4550-abc8-5b1b13a510ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c28b2c8bf424d938d875a1af9a7608c)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:44.025228 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c19be-99a6-4550-abc8-5b1b13a510ae admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:44.026824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894a523-9904-42db-91d5-6010a832a931 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:44.031304 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894a523-9904-42db-91d5-6010a832a931 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:44.032234 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894a523-9904-42db-91d5-6010a832a931 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:44.036452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894a523-9904-42db-91d5-6010a832a931 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:44.102154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894a523-9904-42db-91d5-6010a832a931 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:44.105748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894a523-9904-42db-91d5-6010a832a931 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:44.190430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 659/1335] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:44.286079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-b24c19be-99a6-4550-abc8-5b1b13a510ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c28b2c8bf424d938d875a1af9a7608c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:44.349618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-b24c19be-99a6-4550-abc8-5b1b13a510ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c28b2c8bf424d938d875a1af9a7608c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:44.350962 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-b24c19be-99a6-4550-abc8-5b1b13a510ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c28b2c8bf424d938d875a1af9a7608c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:44.352294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-b24c19be-99a6-4550-abc8-5b1b13a510ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c28b2c8bf424d938d875a1af9a7608c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:44.403728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-b24c19be-99a6-4550-abc8-5b1b13a510ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c28b2c8bf424d938d875a1af9a7608c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:44.522380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 677/1336] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:48:43 2022] DELETE /identity/v3/users/2c28b2c8bf424d938d875a1af9a7608c => generated 0 bytes in 859 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:44.528297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddee09-1739-44e0-ab8e-55f6f50338f4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:44.529672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddee09-1739-44e0-ab8e-55f6f50338f4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:44.530165 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddee09-1739-44e0-ab8e-55f6f50338f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:44.658931 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-bfddee09-1739-44e0-ab8e-55f6f50338f4 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:44.681023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bfddee09-1739-44e0-ab8e-55f6f50338f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:44.721940 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c19be-99a6-4550-abc8-5b1b13a510ae admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:44.728794 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b24c19be-99a6-4550-abc8-5b1b13a510ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:44.798105 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:45.003177 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c19be-99a6-4550-abc8-5b1b13a510ae admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:45.004209 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 660/1337] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:45.013755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84651da1-e787-4a08-88f8-1fc572c16cad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:45.015389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84651da1-e787-4a08-88f8-1fc572c16cad service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:45.016321 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84651da1-e787-4a08-88f8-1fc572c16cad service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:45.017198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84651da1-e787-4a08-88f8-1fc572c16cad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:45.021331 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-84651da1-e787-4a08-88f8-1fc572c16cad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:45.087465 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddee09-1739-44e0-ab8e-55f6f50338f4 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:45.090004 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bfddee09-1739-44e0-ab8e-55f6f50338f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:45.271403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84651da1-e787-4a08-88f8-1fc572c16cad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:45.277941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84651da1-e787-4a08-88f8-1fc572c16cad service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:45.323052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 678/1338] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:45.337921 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddee09-1739-44e0-ab8e-55f6f50338f4 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:45.357609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223b5a5-8b6a-4019-a9bd-461379fac671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:45.359101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223b5a5-8b6a-4019-a9bd-461379fac671 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:45.359867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223b5a5-8b6a-4019-a9bd-461379fac671 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:45.360630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223b5a5-8b6a-4019-a9bd-461379fac671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:45.406453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223b5a5-8b6a-4019-a9bd-461379fac671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:45.411383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223b5a5-8b6a-4019-a9bd-461379fac671 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:45.463479 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 661/1339] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:48:46.799199 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84651da1-e787-4a08-88f8-1fc572c16cad service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:46.801593 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-84651da1-e787-4a08-88f8-1fc572c16cad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:46.839919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84651da1-e787-4a08-88f8-1fc572c16cad service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:46.877946 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:46.884683 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:46.885381 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:46.886044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] PATH_INFO: `/v3/projects/0c29d86a22514388bd9465a4e2fb9daa` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:46.892066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c29d86a22514388bd9465a4e2fb9daa)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:46.901157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:47.070736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c29d86a22514388bd9465a4e2fb9daa', 'request_id': 'req-f957c090-24e9-4d8a-87c0-fc48342ab67e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:47.105455 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c29d86a22514388bd9465a4e2fb9daa', 'request_id': 'req-f957c090-24e9-4d8a-87c0-fc48342ab67e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:47.165773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c29d86a22514388bd9465a4e2fb9daa', 'request_id': 'req-f957c090-24e9-4d8a-87c0-fc48342ab67e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:47.240202 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] The token cache is being invalidate because project 0c29d86a22514388bd9465a4e2fb9daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:48:47.240818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:47.248818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:47.252458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 679/1340] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:48:46 2022] DELETE /identity/v3/projects/0c29d86a22514388bd9465a4e2fb9daa => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:47.258562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90609b3-575c-43fd-9d1f-44c9d9aba386 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:47.260115 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90609b3-575c-43fd-9d1f-44c9d9aba386 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:47.260740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90609b3-575c-43fd-9d1f-44c9d9aba386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:47.386847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f90609b3-575c-43fd-9d1f-44c9d9aba386 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:47.397739 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f90609b3-575c-43fd-9d1f-44c9d9aba386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:47.658689 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:47.709001 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:47.709691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:47.722567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 662/1341] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:47.738455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d785e2-2830-486f-b072-12113477cc30 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:47.738859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d785e2-2830-486f-b072-12113477cc30 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:47.742212 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d785e2-2830-486f-b072-12113477cc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:47.814157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a6d785e2-2830-486f-b072-12113477cc30 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:47.818538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a6d785e2-2830-486f-b072-12113477cc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:47.836725 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957c090-24e9-4d8a-87c0-fc48342ab67e admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:47.867470 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b385b7-bf76-4046-8192-07120b893422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:47.869886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b385b7-bf76-4046-8192-07120b893422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:47.876029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b385b7-bf76-4046-8192-07120b893422 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:47.879245 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b385b7-bf76-4046-8192-07120b893422 admin admin] PATH_INFO: `/v3/users/4d5c3d2aebd94d34a335d685343fe129` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:47.881268 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 663/1342] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:47.887988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b385b7-bf76-4046-8192-07120b893422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d5c3d2aebd94d34a335d685343fe129)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:47.893486 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d785e2-2830-486f-b072-12113477cc30 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:47.896225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b385b7-bf76-4046-8192-07120b893422 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:47.919398 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d785e2-2830-486f-b072-12113477cc30 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:47.973231 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518ba93-33c7-4a56-8644-29b17712f78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:47.974313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518ba93-33c7-4a56-8644-29b17712f78e admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:47.974789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518ba93-33c7-4a56-8644-29b17712f78e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:47.975264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518ba93-33c7-4a56-8644-29b17712f78e admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:47.976527 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518ba93-33c7-4a56-8644-29b17712f78e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:48:47.977066 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518ba93-33c7-4a56-8644-29b17712f78e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:47.980548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518ba93-33c7-4a56-8644-29b17712f78e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:48.032935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 664/1343] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:48:48.035353 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518ba93-33c7-4a56-8644-29b17712f78e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:48.079779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518ba93-33c7-4a56-8644-29b17712f78e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:48.095661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60679fe3-fcc4-443e-9ebc-ab9922a0955c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:48.096942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60679fe3-fcc4-443e-9ebc-ab9922a0955c admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:48.097913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60679fe3-fcc4-443e-9ebc-ab9922a0955c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:48.098489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60679fe3-fcc4-443e-9ebc-ab9922a0955c admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:48.102595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60679fe3-fcc4-443e-9ebc-ab9922a0955c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:48.110708 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60679fe3-fcc4-443e-9ebc-ab9922a0955c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:48.171590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-00b385b7-bf76-4046-8192-07120b893422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d5c3d2aebd94d34a335d685343fe129'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:48.174202 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:48.202873 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-00b385b7-bf76-4046-8192-07120b893422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d5c3d2aebd94d34a335d685343fe129'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:48.204120 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-00b385b7-bf76-4046-8192-07120b893422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d5c3d2aebd94d34a335d685343fe129'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:48.204997 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-00b385b7-bf76-4046-8192-07120b893422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d5c3d2aebd94d34a335d685343fe129'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:48.228917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-00b385b7-bf76-4046-8192-07120b893422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d5c3d2aebd94d34a335d685343fe129'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:48.262340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 665/1344] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:48 2022] POST /identity/v3/projects => generated 383 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:48:48.267971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60679fe3-fcc4-443e-9ebc-ab9922a0955c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:48.275728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-60679fe3-fcc4-443e-9ebc-ab9922a0955c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:48.279605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 680/1345] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:48:47 2022] DELETE /identity/v3/users/4d5c3d2aebd94d34a335d685343fe129 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:48.288775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75c90c-96e7-40e2-a5e4-273fc9e3ada0 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:48.289044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75c90c-96e7-40e2-a5e4-273fc9e3ada0 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:48.289328 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75c90c-96e7-40e2-a5e4-273fc9e3ada0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:48.346085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-2c75c90c-96e7-40e2-a5e4-273fc9e3ada0 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:48.347771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2c75c90c-96e7-40e2-a5e4-273fc9e3ada0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:48.590708 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60679fe3-fcc4-443e-9ebc-ab9922a0955c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:48.599450 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c47b94-4021-44ea-87a3-9f3a42d377c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:48.600161 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c47b94-4021-44ea-87a3-9f3a42d377c0 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:48.600410 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c47b94-4021-44ea-87a3-9f3a42d377c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:48.601067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c47b94-4021-44ea-87a3-9f3a42d377c0 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:48.601912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c47b94-4021-44ea-87a3-9f3a42d377c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:48.604110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 681/1346] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:48:48.608830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c47b94-4021-44ea-87a3-9f3a42d377c0 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:48.614851 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9786a-4e4e-43d9-b4e5-b77fe5e9a38b None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:48.615506 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9786a-4e4e-43d9-b4e5-b77fe5e9a38b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:48.615817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9786a-4e4e-43d9-b4e5-b77fe5e9a38b None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:48.618802 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 682/1347] 10.222.0.36 () {54 vars in 921 bytes} [Sat Oct 15 13:48:48 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:48:48.658550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 666/1348] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:48 2022] POST /identity/v3/users => generated 292 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:48.665418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9786a-4e4e-43d9-b4e5-b77fe5e9a38b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:48.692513 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c47b94-4021-44ea-87a3-9f3a42d377c0 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:48.698508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f6c47b94-4021-44ea-87a3-9f3a42d377c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:48.704812 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9786a-4e4e-43d9-b4e5-b77fe5e9a38b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:48.720820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b881e5-171d-4edf-993c-7b1f63d94e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:48.722117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b881e5-171d-4edf-993c-7b1f63d94e32 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:48.722669 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b881e5-171d-4edf-993c-7b1f63d94e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:48.723291 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b881e5-171d-4edf-993c-7b1f63d94e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:48.724691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b881e5-171d-4edf-993c-7b1f63d94e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:48.728358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b881e5-171d-4edf-993c-7b1f63d94e32 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:48.767965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 683/1349] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:48.769386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b881e5-171d-4edf-993c-7b1f63d94e32 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:48.817329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b881e5-171d-4edf-993c-7b1f63d94e32 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:48.838203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc1690-ec93-459d-ad42-c21493cc9195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:48.842510 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1690-ec93-459d-ad42-c21493cc9195 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:48.844722 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1690-ec93-459d-ad42-c21493cc9195 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:48.845598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1690-ec93-459d-ad42-c21493cc9195 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:48.848021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc1690-ec93-459d-ad42-c21493cc9195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:48.851909 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc1690-ec93-459d-ad42-c21493cc9195 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:48.895113 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 684/1350] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:48.902203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1eba2-b93e-4212-a13a-5f91de619602 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:48.903071 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1eba2-b93e-4212-a13a-5f91de619602 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:48.903841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1eba2-b93e-4212-a13a-5f91de619602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:48.931780 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c47b94-4021-44ea-87a3-9f3a42d377c0 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:48.951941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51bbf5-4f0d-4e37-a8a4-1e32c79bb653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:48.953133 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51bbf5-4f0d-4e37-a8a4-1e32c79bb653 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:48.953853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51bbf5-4f0d-4e37-a8a4-1e32c79bb653 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:48.954543 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51bbf5-4f0d-4e37-a8a4-1e32c79bb653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:48.991360 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-27b1eba2-b93e-4212-a13a-5f91de619602 None None] MFA Rules not processed for user `1c6720518c03413aa8f85e4d6142fe87`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:49.000557 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-27b1eba2-b93e-4212-a13a-5f91de619602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:49.005836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51bbf5-4f0d-4e37-a8a4-1e32c79bb653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:49.015071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51bbf5-4f0d-4e37-a8a4-1e32c79bb653 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:49.056764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 667/1351] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:48:49.062039 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51bbf5-4f0d-4e37-a8a4-1e32c79bb653 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:49.115629 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51bbf5-4f0d-4e37-a8a4-1e32c79bb653 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:49.136055 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ba020-b532-4260-b6a3-34cfa5a95710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:49.137254 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ba020-b532-4260-b6a3-34cfa5a95710 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:49.137734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ba020-b532-4260-b6a3-34cfa5a95710 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:49.138365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ba020-b532-4260-b6a3-34cfa5a95710 admin admin] PATH_INFO: `/v3/projects/f2afb4e61d9e4a5bb60826ab3499c8a2/users/4a56739438b34526b0560443adf3433b/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:49.195466 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ba020-b532-4260-b6a3-34cfa5a95710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2afb4e61d9e4a5bb60826ab3499c8a2, user_id=4a56739438b34526b0560443adf3433b, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:49.203897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ba020-b532-4260-b6a3-34cfa5a95710 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:49.269063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 668/1352] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:49 2022] PUT /identity/v3/projects/f2afb4e61d9e4a5bb60826ab3499c8a2/users/4a56739438b34526b0560443adf3433b/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:49.277375 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248d13c-45be-4865-a5a1-2bc6f6fb13d5 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:49.277861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248d13c-45be-4865-a5a1-2bc6f6fb13d5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:49.282662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248d13c-45be-4865-a5a1-2bc6f6fb13d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:49.299509 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 685/1353] 10.222.0.36 () {58 vars in 1018 bytes} [Sat Oct 15 13:48:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:49.332755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1eba2-b93e-4212-a13a-5f91de619602 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:49.334319 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-27b1eba2-b93e-4212-a13a-5f91de619602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:49.355416 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1eba2-b93e-4212-a13a-5f91de619602 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:49.383930 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-1248d13c-45be-4865-a5a1-2bc6f6fb13d5 None None] MFA Rules not processed for user `4a56739438b34526b0560443adf3433b`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:49.386433 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1248d13c-45be-4865-a5a1-2bc6f6fb13d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:49.419960 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:49.481466 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:49.610833 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f9f26-8d12-4f29-bfdc-d6d874246e85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6720518c03413aa8f85e4d6142fe87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:49.611765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f9f26-8d12-4f29-bfdc-d6d874246e85 service placement] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:49.612075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f9f26-8d12-4f29-bfdc-d6d874246e85 service placement] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:49.612513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f9f26-8d12-4f29-bfdc-d6d874246e85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:49.653977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f9f26-8d12-4f29-bfdc-d6d874246e85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:49.659422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f9f26-8d12-4f29-bfdc-d6d874246e85 service placement] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:49.689981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 686/1354] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:48:49.707998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f9f26-8d12-4f29-bfdc-d6d874246e85 service placement] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:49.710753 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 669/1355] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:49.917376 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f9f26-8d12-4f29-bfdc-d6d874246e85 service placement] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:49.934571 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876431fe-e6d2-45ad-b4ec-1eca0e45e8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:49.935367 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876431fe-e6d2-45ad-b4ec-1eca0e45e8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:49.935781 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876431fe-e6d2-45ad-b4ec-1eca0e45e8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:49.936281 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876431fe-e6d2-45ad-b4ec-1eca0e45e8f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:49.939304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876431fe-e6d2-45ad-b4ec-1eca0e45e8f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:49.942939 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876431fe-e6d2-45ad-b4ec-1eca0e45e8f2 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:50.009625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 687/1356] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:49 2022] POST /identity/v3/projects => generated 383 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:48:50.016871 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248d13c-45be-4865-a5a1-2bc6f6fb13d5 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:50.077180 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248d13c-45be-4865-a5a1-2bc6f6fb13d5 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:50.350880 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64fbcc-a961-4be2-92fe-be38002b4363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:50.352226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64fbcc-a961-4be2-92fe-be38002b4363 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:50.353028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64fbcc-a961-4be2-92fe-be38002b4363 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:50.353481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64fbcc-a961-4be2-92fe-be38002b4363 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:50.356213 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64fbcc-a961-4be2-92fe-be38002b4363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:50.368638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64fbcc-a961-4be2-92fe-be38002b4363 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:50.479080 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 670/1357] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:50 2022] POST /identity/v3/users => generated 298 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:50.485894 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876431fe-e6d2-45ad-b4ec-1eca0e45e8f2 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:50.509301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876431fe-e6d2-45ad-b4ec-1eca0e45e8f2 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:50.520824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c91e1-ada3-4a4d-ade2-3503d66be847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:50.521541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c91e1-ada3-4a4d-ade2-3503d66be847 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:50.521826 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c91e1-ada3-4a4d-ade2-3503d66be847 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:50.522183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c91e1-ada3-4a4d-ade2-3503d66be847 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:50.522870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c91e1-ada3-4a4d-ade2-3503d66be847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:50.526416 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c91e1-ada3-4a4d-ade2-3503d66be847 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:50.620188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 688/1358] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:48:50.626646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64fbcc-a961-4be2-92fe-be38002b4363 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:50.646009 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64fbcc-a961-4be2-92fe-be38002b4363 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:50.668032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8c145-66e4-4d2d-826a-a61c80f4e677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:50.670325 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8c145-66e4-4d2d-826a-a61c80f4e677 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:50.670897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8c145-66e4-4d2d-826a-a61c80f4e677 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:50.672302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8c145-66e4-4d2d-826a-a61c80f4e677 admin admin] PATH_INFO: `/v3/projects/dabf57723d854192a1782ad10b212e2e/users/e3f000375aa24b209f9f89c30782bf3b/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:50.725833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8c145-66e4-4d2d-826a-a61c80f4e677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dabf57723d854192a1782ad10b212e2e, user_id=e3f000375aa24b209f9f89c30782bf3b, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:50.729252 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce8c145-66e4-4d2d-826a-a61c80f4e677 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:50.761271 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 671/1359] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:50 2022] PUT /identity/v3/projects/dabf57723d854192a1782ad10b212e2e/users/e3f000375aa24b209f9f89c30782bf3b/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:50.767631 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c91e1-ada3-4a4d-ade2-3503d66be847 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:50.789645 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c91e1-ada3-4a4d-ade2-3503d66be847 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:50.948049 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ccd44-0acc-4303-acd0-9b7b85714243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:50.949111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ccd44-0acc-4303-acd0-9b7b85714243 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:50.949111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ccd44-0acc-4303-acd0-9b7b85714243 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:50.949430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ccd44-0acc-4303-acd0-9b7b85714243 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:50.949920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ccd44-0acc-4303-acd0-9b7b85714243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:50.951988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ccd44-0acc-4303-acd0-9b7b85714243 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:50.979485 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 689/1360] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:50.983464 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8c145-66e4-4d2d-826a-a61c80f4e677 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:50.995289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8c145-66e4-4d2d-826a-a61c80f4e677 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:51.013311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065bc73-60a8-416e-90b5-dc6e64c2bb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:51.014038 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065bc73-60a8-416e-90b5-dc6e64c2bb84 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:51.014361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065bc73-60a8-416e-90b5-dc6e64c2bb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:51.014759 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065bc73-60a8-416e-90b5-dc6e64c2bb84 admin admin] PATH_INFO: `/v3/projects/dabf57723d854192a1782ad10b212e2e/users/e3f000375aa24b209f9f89c30782bf3b/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:51.021560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065bc73-60a8-416e-90b5-dc6e64c2bb84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dabf57723d854192a1782ad10b212e2e, user_id=e3f000375aa24b209f9f89c30782bf3b, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:51.024463 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065bc73-60a8-416e-90b5-dc6e64c2bb84 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:51.055951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 672/1361] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:50 2022] PUT /identity/v3/projects/dabf57723d854192a1782ad10b212e2e/users/e3f000375aa24b209f9f89c30782bf3b/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:48:51.076578 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76226f9-92fe-430e-a3a2-08668bda466f None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:51.077219 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76226f9-92fe-430e-a3a2-08668bda466f None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:51.077658 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76226f9-92fe-430e-a3a2-08668bda466f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:51.151455 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-c76226f9-92fe-430e-a3a2-08668bda466f None None] MFA Rules not processed for user `e3f000375aa24b209f9f89c30782bf3b`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:51.155194 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c76226f9-92fe-430e-a3a2-08668bda466f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:51.255596 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a89ed-d60d-48a9-9c12-2cf3963a9dc8 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:51.256159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a89ed-d60d-48a9-9c12-2cf3963a9dc8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:51.256772 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a89ed-d60d-48a9-9c12-2cf3963a9dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:51.275402 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:51.419671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 690/1362] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:48:51.419671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76226f9-92fe-430e-a3a2-08668bda466f None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:51.419671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c76226f9-92fe-430e-a3a2-08668bda466f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:51.451964 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-560a89ed-d60d-48a9-9c12-2cf3963a9dc8 None None] MFA Rules not processed for user `5150c06020594efe957359f641eba5da`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:51.460374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-560a89ed-d60d-48a9-9c12-2cf3963a9dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:51.734318 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76226f9-92fe-430e-a3a2-08668bda466f None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:51.736578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:51.792111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152e820-3c6c-4ebe-90e2-ae67f6efcd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:51.795234 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152e820-3c6c-4ebe-90e2-ae67f6efcd0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:51.800259 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152e820-3c6c-4ebe-90e2-ae67f6efcd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:51.800868 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152e820-3c6c-4ebe-90e2-ae67f6efcd0f admin admin] PATH_INFO: `/v3/users/667cdddd4e314f60b8b2a6ddc64152df` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:51.805748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152e820-3c6c-4ebe-90e2-ae67f6efcd0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=667cdddd4e314f60b8b2a6ddc64152df)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:51.820401 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152e820-3c6c-4ebe-90e2-ae67f6efcd0f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:51.822021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 673/1363] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 570 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:51.823962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a89ed-d60d-48a9-9c12-2cf3963a9dc8 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:51.827855 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-560a89ed-d60d-48a9-9c12-2cf3963a9dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:52.009673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a89ed-d60d-48a9-9c12-2cf3963a9dc8 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:52.024723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16501df8-5791-4d6d-8934-d73a1030d565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:52.025397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16501df8-5791-4d6d-8934-d73a1030d565 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:52.025669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16501df8-5791-4d6d-8934-d73a1030d565 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:52.026147 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16501df8-5791-4d6d-8934-d73a1030d565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:52.033815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c152e820-3c6c-4ebe-90e2-ae67f6efcd0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '667cdddd4e314f60b8b2a6ddc64152df'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:52.044659 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16501df8-5791-4d6d-8934-d73a1030d565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:52.046192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16501df8-5791-4d6d-8934-d73a1030d565 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:52.049894 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c152e820-3c6c-4ebe-90e2-ae67f6efcd0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '667cdddd4e314f60b8b2a6ddc64152df'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:52.061211 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c152e820-3c6c-4ebe-90e2-ae67f6efcd0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '667cdddd4e314f60b8b2a6ddc64152df'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:52.062301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c152e820-3c6c-4ebe-90e2-ae67f6efcd0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '667cdddd4e314f60b8b2a6ddc64152df'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:52.072587 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c152e820-3c6c-4ebe-90e2-ae67f6efcd0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '667cdddd4e314f60b8b2a6ddc64152df'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:52.074912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 674/1364] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:52.076276 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16501df8-5791-4d6d-8934-d73a1030d565 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:52.089502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-16501df8-5791-4d6d-8934-d73a1030d565 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:52.129029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16501df8-5791-4d6d-8934-d73a1030d565 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:52.146337 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 691/1365] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:48:51 2022] DELETE /identity/v3/users/667cdddd4e314f60b8b2a6ddc64152df => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:52.147981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2c148-5bce-4ced-ae7c-c2b201470c1d None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:52.148274 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2c148-5bce-4ced-ae7c-c2b201470c1d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:52.148552 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2c148-5bce-4ced-ae7c-c2b201470c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:52.153785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:52.153785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:52.153785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:52.153785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] PATH_INFO: `/v3/projects/735c94e76c1e4aca8c1b82ea5d744e62` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:52.164709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=735c94e76c1e4aca8c1b82ea5d744e62)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:52.168333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 15 13:48:52.168333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Oct 15 13:48:52.168333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: Oct 15 13:48:52.179636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:52.211286 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-e0f2c148-5bce-4ced-ae7c-c2b201470c1d None None] MFA Rules not processed for user `5150c06020594efe957359f641eba5da`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:52.212597 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e0f2c148-5bce-4ced-ae7c-c2b201470c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:52.381197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:52.424993 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 692/1366] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:52.426755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff12e9-d868-49b6-8578-813a81203c42 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:52.427108 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff12e9-d868-49b6-8578-813a81203c42 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:52.427480 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff12e9-d868-49b6-8578-813a81203c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:52.471728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '735c94e76c1e4aca8c1b82ea5d744e62', 'request_id': 'req-67ab4a75-2c28-4804-9172-58ef77c50ab9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:52.538364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-3eff12e9-d868-49b6-8578-813a81203c42 None None] MFA Rules not processed for user `583cce1547144e11978f20a3193ceba4`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:52.542468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3eff12e9-d868-49b6-8578-813a81203c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:52.542797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '735c94e76c1e4aca8c1b82ea5d744e62', 'request_id': 'req-67ab4a75-2c28-4804-9172-58ef77c50ab9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:52.579584 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '735c94e76c1e4aca8c1b82ea5d744e62', 'request_id': 'req-67ab4a75-2c28-4804-9172-58ef77c50ab9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:52.641266 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] The token cache is being invalidate because project 735c94e76c1e4aca8c1b82ea5d744e62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:48:52.642148 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:52.643501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:52.652601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 675/1367] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:48:52 2022] DELETE /identity/v3/projects/735c94e76c1e4aca8c1b82ea5d744e62 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:52.655773 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:52.676512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:52.775699 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 693/1368] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:52.780210 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff12e9-d868-49b6-8578-813a81203c42 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:52.791772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3eff12e9-d868-49b6-8578-813a81203c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:53.030692 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab4a75-2c28-4804-9172-58ef77c50ab9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:53.038961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73f1df-8d02-4462-b699-b58f5faaa7ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:53.039796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73f1df-8d02-4462-b699-b58f5faaa7ae service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:53.040333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73f1df-8d02-4462-b699-b58f5faaa7ae service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:53.040652 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73f1df-8d02-4462-b699-b58f5faaa7ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:53.052324 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bc73f1df-8d02-4462-b699-b58f5faaa7ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:53.148140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff12e9-d868-49b6-8578-813a81203c42 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:53.154547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:53.164364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbed623-7aaf-43e2-b2dc-0b6fff1ce3e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:53.166294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbed623-7aaf-43e2-b2dc-0b6fff1ce3e6 service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:53.166925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbed623-7aaf-43e2-b2dc-0b6fff1ce3e6 service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:53.167736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbed623-7aaf-43e2-b2dc-0b6fff1ce3e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:53.174983 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1dbed623-7aaf-43e2-b2dc-0b6fff1ce3e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:53.216548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73f1df-8d02-4462-b699-b58f5faaa7ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:53.218069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73f1df-8d02-4462-b699-b58f5faaa7ae service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:53.245825 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 676/1369] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:53.249392 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3c783-4694-4087-abbd-305e739cfb3b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:53.249711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3c783-4694-4087-abbd-305e739cfb3b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:53.252144 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3c783-4694-4087-abbd-305e739cfb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:53.298299 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-eec3c783-4694-4087-abbd-305e739cfb3b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:53.304211 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-eec3c783-4694-4087-abbd-305e739cfb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:53.389258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbed623-7aaf-43e2-b2dc-0b6fff1ce3e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:53.392247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbed623-7aaf-43e2-b2dc-0b6fff1ce3e6 service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:53.415161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 694/1370] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:53.417029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbed623-7aaf-43e2-b2dc-0b6fff1ce3e6 service nova] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:53.420915 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1dbed623-7aaf-43e2-b2dc-0b6fff1ce3e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:53.456571 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbed623-7aaf-43e2-b2dc-0b6fff1ce3e6 service nova] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:53.471324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce1ead-999e-43d7-bfd7-97f92ed1c77f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:53.472625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce1ead-999e-43d7-bfd7-97f92ed1c77f service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:53.473198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce1ead-999e-43d7-bfd7-97f92ed1c77f service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:53.473938 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce1ead-999e-43d7-bfd7-97f92ed1c77f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:53.509457 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce1ead-999e-43d7-bfd7-97f92ed1c77f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:53.509846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 677/1371] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:53.511594 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3c783-4694-4087-abbd-305e739cfb3b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:53.513455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-eec3c783-4694-4087-abbd-305e739cfb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:53.514597 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce1ead-999e-43d7-bfd7-97f92ed1c77f service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:53.530259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3c783-4694-4087-abbd-305e739cfb3b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:53.546506 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:53.548155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:53.551201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:53.551735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] PATH_INFO: `/v3/projects/2f75462e70f94e8a8ee4d907be539416` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:53.555969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 695/1372] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:53.556307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f75462e70f94e8a8ee4d907be539416)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:53.560920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b8250-3fc0-4660-a91d-698ff2ed6d7f None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:53.561539 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b8250-3fc0-4660-a91d-698ff2ed6d7f None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:53.561712 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:53.562153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b8250-3fc0-4660-a91d-698ff2ed6d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:53.604071 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:53.699703 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f75462e70f94e8a8ee4d907be539416', 'request_id': 'req-97161ef1-5679-4b90-87c4-ef8c122f095a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:53.756107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f75462e70f94e8a8ee4d907be539416', 'request_id': 'req-97161ef1-5679-4b90-87c4-ef8c122f095a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:53.780033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f75462e70f94e8a8ee4d907be539416', 'request_id': 'req-97161ef1-5679-4b90-87c4-ef8c122f095a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:53.826025 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] The token cache is being invalidate because project 2f75462e70f94e8a8ee4d907be539416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:48:53.826613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:53.827641 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:53.832601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 678/1373] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:48:53 2022] DELETE /identity/v3/projects/2f75462e70f94e8a8ee4d907be539416 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:53.838805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:53.839169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-794b8250-3fc0-4660-a91d-698ff2ed6d7f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:53.840392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-794b8250-3fc0-4660-a91d-698ff2ed6d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:53.841849 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:53.984249 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 696/1374] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:54.001625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b8250-3fc0-4660-a91d-698ff2ed6d7f None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:54.030631 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b8250-3fc0-4660-a91d-698ff2ed6d7f None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:54.066032 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ded87-341a-4a60-88c5-e3b918abe9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:54.067089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ded87-341a-4a60-88c5-e3b918abe9ef admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:54.067558 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ded87-341a-4a60-88c5-e3b918abe9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:54.071797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ded87-341a-4a60-88c5-e3b918abe9ef admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:54.073073 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ded87-341a-4a60-88c5-e3b918abe9ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:48:54.073575 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ded87-341a-4a60-88c5-e3b918abe9ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:54.094579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ded87-341a-4a60-88c5-e3b918abe9ef admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:54.119677 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:54.142379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97161ef1-5679-4b90-87c4-ef8c122f095a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:54.150556 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 697/1375] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:48:54.153932 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ded87-341a-4a60-88c5-e3b918abe9ef admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:54.166516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6b1ae-ebae-4831-bbe2-4d43c6923e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:54.167652 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6b1ae-ebae-4831-bbe2-4d43c6923e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:54.168251 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6b1ae-ebae-4831-bbe2-4d43c6923e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:54.168723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6b1ae-ebae-4831-bbe2-4d43c6923e5c admin admin] PATH_INFO: `/v3/users/824a4131151746288637abade5ab9dfd` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:54.173691 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6b1ae-ebae-4831-bbe2-4d43c6923e5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=824a4131151746288637abade5ab9dfd)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:54.176870 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6b1ae-ebae-4831-bbe2-4d43c6923e5c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:54.200134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ded87-341a-4a60-88c5-e3b918abe9ef admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:54.208875 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e575318-3376-4664-b75c-aa50de30cfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:54.209897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e575318-3376-4664-b75c-aa50de30cfef admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:54.210569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e575318-3376-4664-b75c-aa50de30cfef admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:54.211121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e575318-3376-4664-b75c-aa50de30cfef admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:54.212370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e575318-3376-4664-b75c-aa50de30cfef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:54.216120 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e575318-3376-4664-b75c-aa50de30cfef admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:54.298178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 698/1376] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:54 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:54.299102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e575318-3376-4664-b75c-aa50de30cfef admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:54.301631 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2e575318-3376-4664-b75c-aa50de30cfef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:54.335560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8eb6b1ae-ebae-4831-bbe2-4d43c6923e5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '824a4131151746288637abade5ab9dfd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:54.336596 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8eb6b1ae-ebae-4831-bbe2-4d43c6923e5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '824a4131151746288637abade5ab9dfd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:54.339197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8eb6b1ae-ebae-4831-bbe2-4d43c6923e5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '824a4131151746288637abade5ab9dfd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:54.391722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8eb6b1ae-ebae-4831-bbe2-4d43c6923e5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '824a4131151746288637abade5ab9dfd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:54.408012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8eb6b1ae-ebae-4831-bbe2-4d43c6923e5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '824a4131151746288637abade5ab9dfd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:54.438105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 679/1377] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:48:53 2022] DELETE /identity/v3/users/824a4131151746288637abade5ab9dfd => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:48:54.443657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6b1ae-ebae-4831-bbe2-4d43c6923e5c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:54.450628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8eb6b1ae-ebae-4831-bbe2-4d43c6923e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:54.524587 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e575318-3376-4664-b75c-aa50de30cfef admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:54.678018 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ce526-64af-4c51-a058-03d45ff4a398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:54.678576 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ce526-64af-4c51-a058-03d45ff4a398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:54.679796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ce526-64af-4c51-a058-03d45ff4a398 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:54.680167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ce526-64af-4c51-a058-03d45ff4a398 admin admin] PATH_INFO: `/v3/users/4a56739438b34526b0560443adf3433b` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:54.687824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ce526-64af-4c51-a058-03d45ff4a398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a56739438b34526b0560443adf3433b)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:54.691269 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ce526-64af-4c51-a058-03d45ff4a398 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:54.742139 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:54.781188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6b1ae-ebae-4831-bbe2-4d43c6923e5c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:54.800372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be28589-daca-4d63-bbc0-b03ac2580af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:54.801457 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be28589-daca-4d63-bbc0-b03ac2580af2 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:54.801917 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be28589-daca-4d63-bbc0-b03ac2580af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:54.802573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be28589-daca-4d63-bbc0-b03ac2580af2 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:54.806435 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be28589-daca-4d63-bbc0-b03ac2580af2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:54.807248 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-1d0ce526-64af-4c51-a058-03d45ff4a398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a56739438b34526b0560443adf3433b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:54.810814 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be28589-daca-4d63-bbc0-b03ac2580af2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:54.827159 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-1d0ce526-64af-4c51-a058-03d45ff4a398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a56739438b34526b0560443adf3433b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:54.828268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-1d0ce526-64af-4c51-a058-03d45ff4a398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a56739438b34526b0560443adf3433b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:54.832564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-1d0ce526-64af-4c51-a058-03d45ff4a398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a56739438b34526b0560443adf3433b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:54.854347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-1d0ce526-64af-4c51-a058-03d45ff4a398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a56739438b34526b0560443adf3433b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:54.903683 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 699/1378] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:48:54 2022] DELETE /identity/v3/users/4a56739438b34526b0560443adf3433b => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:54.914158 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8282259-4f8e-4e92-927c-08a13d489da2 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:54.915044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8282259-4f8e-4e92-927c-08a13d489da2 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:54.915785 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8282259-4f8e-4e92-927c-08a13d489da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:54.946279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 680/1379] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:54 2022] POST /identity/v3/users => generated 292 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:54.956230 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be28589-daca-4d63-bbc0-b03ac2580af2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:54.960251 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9be28589-daca-4d63-bbc0-b03ac2580af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:55.010653 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-a8282259-4f8e-4e92-927c-08a13d489da2 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:55.012212 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-a8282259-4f8e-4e92-927c-08a13d489da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:55.035081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be28589-daca-4d63-bbc0-b03ac2580af2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:55.200429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d1571-5420-4363-abe6-449916a09fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:55.200981 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d1571-5420-4363-abe6-449916a09fbf admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:55.201316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d1571-5420-4363-abe6-449916a09fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:55.201598 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d1571-5420-4363-abe6-449916a09fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:55.202500 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d1571-5420-4363-abe6-449916a09fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:55.205698 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d1571-5420-4363-abe6-449916a09fbf admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:55.223456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:55.229434 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 700/1380] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:55.237950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f805ff7-7db1-4dd8-bbf5-ba2488c7770e None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:55.240797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 681/1381] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:55.240954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f805ff7-7db1-4dd8-bbf5-ba2488c7770e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:55.242706 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d1571-5420-4363-abe6-449916a09fbf admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:55.243100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f805ff7-7db1-4dd8-bbf5-ba2488c7770e None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:55.255778 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 701/1382] 10.222.0.36 () {54 vars in 912 bytes} [Sat Oct 15 13:48:55 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:48:55.257804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f805ff7-7db1-4dd8-bbf5-ba2488c7770e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:55.261039 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-5f805ff7-7db1-4dd8-bbf5-ba2488c7770e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:55.287484 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d1571-5420-4363-abe6-449916a09fbf admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:55.304812 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9de900-5219-4f40-9dcb-f95666a2ef83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:55.308806 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9de900-5219-4f40-9dcb-f95666a2ef83 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:55.308990 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9de900-5219-4f40-9dcb-f95666a2ef83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:55.309351 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9de900-5219-4f40-9dcb-f95666a2ef83 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:55.310192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9de900-5219-4f40-9dcb-f95666a2ef83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:55.312903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9de900-5219-4f40-9dcb-f95666a2ef83 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:55.334989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 682/1383] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:55.334989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a02001-897f-4e19-acf3-5b1cd00d3059 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:55.334989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a02001-897f-4e19-acf3-5b1cd00d3059 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:55.334989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a02001-897f-4e19-acf3-5b1cd00d3059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:55.488940 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-91a02001-897f-4e19-acf3-5b1cd00d3059 None None] MFA Rules not processed for user `90985ec8c43a4d7895942731dc1d002c`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:55.491030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-91a02001-897f-4e19-acf3-5b1cd00d3059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:55.542229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f805ff7-7db1-4dd8-bbf5-ba2488c7770e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:55.563713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95506ca4-e53b-4783-a9ce-77c4665a95e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:55.563713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95506ca4-e53b-4783-a9ce-77c4665a95e9 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:55.563713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95506ca4-e53b-4783-a9ce-77c4665a95e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:55.563713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95506ca4-e53b-4783-a9ce-77c4665a95e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:55.608276 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95506ca4-e53b-4783-a9ce-77c4665a95e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:55.611707 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95506ca4-e53b-4783-a9ce-77c4665a95e9 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:55.655921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 702/1384] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:55.657783 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95506ca4-e53b-4783-a9ce-77c4665a95e9 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:55.717030 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95506ca4-e53b-4783-a9ce-77c4665a95e9 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:55.744103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 683/1385] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:48:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:55.748529 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a02001-897f-4e19-acf3-5b1cd00d3059 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:55.757134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa21b9-4e6d-4c4f-94b7-c53ee9be61ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:55.758282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa21b9-4e6d-4c4f-94b7-c53ee9be61ef admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:55.759411 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa21b9-4e6d-4c4f-94b7-c53ee9be61ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:55.759411 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa21b9-4e6d-4c4f-94b7-c53ee9be61ef admin admin] PATH_INFO: `/v3/projects/048e14a2201e407b92e80c8a758159b7/users/9c1eb56352494cff86f71d80a012d060/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:55.763196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-91a02001-897f-4e19-acf3-5b1cd00d3059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:55.802291 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa21b9-4e6d-4c4f-94b7-c53ee9be61ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=048e14a2201e407b92e80c8a758159b7, user_id=9c1eb56352494cff86f71d80a012d060, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:55.808116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa21b9-4e6d-4c4f-94b7-c53ee9be61ef admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:55.834886 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a02001-897f-4e19-acf3-5b1cd00d3059 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:55.861603 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc46e54-5854-45e0-b309-29a5335c4b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:55.862712 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc46e54-5854-45e0-b309-29a5335c4b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:55.863202 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc46e54-5854-45e0-b309-29a5335c4b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:55.863679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc46e54-5854-45e0-b309-29a5335c4b92 admin admin] PATH_INFO: `/v3/projects/73abe7085bd341f3bca2322efa9f16a7` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:55.889502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc46e54-5854-45e0-b309-29a5335c4b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73abe7085bd341f3bca2322efa9f16a7)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:55.893390 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc46e54-5854-45e0-b309-29a5335c4b92 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:55.917339 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 703/1386] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:55 2022] PUT /identity/v3/projects/048e14a2201e407b92e80c8a758159b7/users/9c1eb56352494cff86f71d80a012d060/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:55.919818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa21b9-4e6d-4c4f-94b7-c53ee9be61ef admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:55.955113 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:56.113942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa21b9-4e6d-4c4f-94b7-c53ee9be61ef admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:56.124927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5e3a8-2987-437c-80e8-06618c2e3478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:56.125961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5e3a8-2987-437c-80e8-06618c2e3478 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:56.128131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5e3a8-2987-437c-80e8-06618c2e3478 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:56.129483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5e3a8-2987-437c-80e8-06618c2e3478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:56.172302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5e3a8-2987-437c-80e8-06618c2e3478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:56.184262 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5e3a8-2987-437c-80e8-06618c2e3478 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:56.198437 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 704/1387] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:56.200392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0fde1-3ae2-41ea-a799-32c12d371aa4 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:56.200908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0fde1-3ae2-41ea-a799-32c12d371aa4 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:56.201348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0fde1-3ae2-41ea-a799-32c12d371aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:56.209409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7dc46e54-5854-45e0-b309-29a5335c4b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73abe7085bd341f3bca2322efa9f16a7', 'request_id': 'req-7dc46e54-5854-45e0-b309-29a5335c4b92'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:56.249409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7dc46e54-5854-45e0-b309-29a5335c4b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73abe7085bd341f3bca2322efa9f16a7', 'request_id': 'req-7dc46e54-5854-45e0-b309-29a5335c4b92'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:56.272300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7dc46e54-5854-45e0-b309-29a5335c4b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73abe7085bd341f3bca2322efa9f16a7', 'request_id': 'req-7dc46e54-5854-45e0-b309-29a5335c4b92'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:56.292158 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-77f0fde1-3ae2-41ea-a799-32c12d371aa4 None None] MFA Rules not processed for user `9c1eb56352494cff86f71d80a012d060`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:56.293739 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-77f0fde1-3ae2-41ea-a799-32c12d371aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:56.307101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7dc46e54-5854-45e0-b309-29a5335c4b92 admin admin] The token cache is being invalidate because project 73abe7085bd341f3bca2322efa9f16a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:48:56.307459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7dc46e54-5854-45e0-b309-29a5335c4b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:56.308457 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7dc46e54-5854-45e0-b309-29a5335c4b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:56.318754 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 684/1388] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:48:55 2022] DELETE /identity/v3/projects/73abe7085bd341f3bca2322efa9f16a7 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:56.321492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cc808-83fe-4ec5-90aa-b08679f43b42 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:56.321492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cc808-83fe-4ec5-90aa-b08679f43b42 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:56.321492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cc808-83fe-4ec5-90aa-b08679f43b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:56.384111 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ef7cc808-83fe-4ec5-90aa-b08679f43b42 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:56.385452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ef7cc808-83fe-4ec5-90aa-b08679f43b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:56.579272 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 705/1389] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:56.584429 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0fde1-3ae2-41ea-a799-32c12d371aa4 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:56.586082 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-77f0fde1-3ae2-41ea-a799-32c12d371aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:56.587850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 685/1390] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:56.608689 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80987706-23fd-432c-8386-5d5e7df70027 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:56.609276 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80987706-23fd-432c-8386-5d5e7df70027 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:56.609678 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80987706-23fd-432c-8386-5d5e7df70027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:56.619156 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0fde1-3ae2-41ea-a799-32c12d371aa4 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:56.630028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c16a9-5b58-4724-b90e-26b76181d364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:56.630693 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c16a9-5b58-4724-b90e-26b76181d364 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:56.630976 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c16a9-5b58-4724-b90e-26b76181d364 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:56.631251 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c16a9-5b58-4724-b90e-26b76181d364 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:56.631950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c16a9-5b58-4724-b90e-26b76181d364 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:56.634031 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c16a9-5b58-4724-b90e-26b76181d364 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:56.688126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-80987706-23fd-432c-8386-5d5e7df70027 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:56.696248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-80987706-23fd-432c-8386-5d5e7df70027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:56.702258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 706/1391] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:56 2022] POST /identity/v3/projects => generated 389 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:56.706631 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c16a9-5b58-4724-b90e-26b76181d364 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:56.715468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:56.862992 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c16a9-5b58-4724-b90e-26b76181d364 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:56.880572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffd2a5-afdf-4b1d-ba3b-4f3cbf939c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:56.880572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffd2a5-afdf-4b1d-ba3b-4f3cbf939c38 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:56.880572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffd2a5-afdf-4b1d-ba3b-4f3cbf939c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:56.880572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffd2a5-afdf-4b1d-ba3b-4f3cbf939c38 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:56.881593 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffd2a5-afdf-4b1d-ba3b-4f3cbf939c38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:56.884873 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffd2a5-afdf-4b1d-ba3b-4f3cbf939c38 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:56.944327 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 707/1392] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:56 2022] POST /identity/v3/users => generated 301 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:48:56.949205 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffd2a5-afdf-4b1d-ba3b-4f3cbf939c38 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:56.955648 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 686/1393] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:56.960640 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80987706-23fd-432c-8386-5d5e7df70027 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:56.981338 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffd2a5-afdf-4b1d-ba3b-4f3cbf939c38 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:56.984630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80987706-23fd-432c-8386-5d5e7df70027 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:57.005164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ee5e6-77da-4b63-b4bf-7babe0ee8422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:57.006757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374edaf-41bc-41fa-aa76-06fe34d18e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:57.007799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374edaf-41bc-41fa-aa76-06fe34d18e3d admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:57.008301 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374edaf-41bc-41fa-aa76-06fe34d18e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:57.008758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374edaf-41bc-41fa-aa76-06fe34d18e3d admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:57.010856 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ee5e6-77da-4b63-b4bf-7babe0ee8422 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:57.011321 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ee5e6-77da-4b63-b4bf-7babe0ee8422 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:57.011806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ee5e6-77da-4b63-b4bf-7babe0ee8422 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:57.013972 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ee5e6-77da-4b63-b4bf-7babe0ee8422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:57.016389 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374edaf-41bc-41fa-aa76-06fe34d18e3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:48:57.016656 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374edaf-41bc-41fa-aa76-06fe34d18e3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:57.019545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374edaf-41bc-41fa-aa76-06fe34d18e3d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:57.023719 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ee5e6-77da-4b63-b4bf-7babe0ee8422 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:57.067011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 708/1394] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:48:57.070947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 687/1395] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:48:57.071540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ee5e6-77da-4b63-b4bf-7babe0ee8422 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:57.077673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374edaf-41bc-41fa-aa76-06fe34d18e3d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:57.079046 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:57.101782 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ee5e6-77da-4b63-b4bf-7babe0ee8422 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:57.116326 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7498d-7450-49c8-a074-d3279b4e6fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:57.117669 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7498d-7450-49c8-a074-d3279b4e6fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:57.118128 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7498d-7450-49c8-a074-d3279b4e6fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:57.118585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7498d-7450-49c8-a074-d3279b4e6fbd admin admin] PATH_INFO: `/v3/projects/cafd5b88d94940b99914be31798f3e57/users/5e76fb7d212d4912b079288aab860d76/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:57.126210 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374edaf-41bc-41fa-aa76-06fe34d18e3d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:57.153082 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530ed90-13b2-4cc5-b9fd-6d556d6c8208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:57.154853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530ed90-13b2-4cc5-b9fd-6d556d6c8208 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:57.155364 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530ed90-13b2-4cc5-b9fd-6d556d6c8208 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:57.155806 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530ed90-13b2-4cc5-b9fd-6d556d6c8208 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:57.157047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530ed90-13b2-4cc5-b9fd-6d556d6c8208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:57.167018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d530ed90-13b2-4cc5-b9fd-6d556d6c8208 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:57.189240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7498d-7450-49c8-a074-d3279b4e6fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cafd5b88d94940b99914be31798f3e57, user_id=5e76fb7d212d4912b079288aab860d76, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:57.230058 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7498d-7450-49c8-a074-d3279b4e6fbd admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:57.265148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 709/1396] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:57 2022] PUT /identity/v3/projects/cafd5b88d94940b99914be31798f3e57/users/5e76fb7d212d4912b079288aab860d76/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:48:57.266385 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7498d-7450-49c8-a074-d3279b4e6fbd admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:57.323188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 688/1397] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:48:57 2022] POST /identity/v3/projects => generated 363 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:48:57.329533 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530ed90-13b2-4cc5-b9fd-6d556d6c8208 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:57.509341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7498d-7450-49c8-a074-d3279b4e6fbd admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:57.622485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d530ed90-13b2-4cc5-b9fd-6d556d6c8208 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:57.646208 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ebab0-f262-4cfb-94aa-c9b687491f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:57.649498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ebab0-f262-4cfb-94aa-c9b687491f2b admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:57.650735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ebab0-f262-4cfb-94aa-c9b687491f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:57.651861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ebab0-f262-4cfb-94aa-c9b687491f2b admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:57.651861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ebab0-f262-4cfb-94aa-c9b687491f2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:57.656182 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ebab0-f262-4cfb-94aa-c9b687491f2b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:57.768676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 689/1398] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:48:57 2022] POST /identity/v3/users => generated 282 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:48:57.770695 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ebab0-f262-4cfb-94aa-c9b687491f2b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:57.779851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4f8ebab0-f262-4cfb-94aa-c9b687491f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:57.810819 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc0530-7675-4b49-803d-e3b97a7a2d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:57.813412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0530-7675-4b49-803d-e3b97a7a2d00 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:57.817433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0530-7675-4b49-803d-e3b97a7a2d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:57.817668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0530-7675-4b49-803d-e3b97a7a2d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:57.820070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc0530-7675-4b49-803d-e3b97a7a2d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:57.826262 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc0530-7675-4b49-803d-e3b97a7a2d00 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:57.866962 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 710/1399] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 601 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:57.868836 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8bb55-3b33-4291-9556-d4243b039136 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:57.875034 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8bb55-3b33-4291-9556-d4243b039136 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:57.878142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8bb55-3b33-4291-9556-d4243b039136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:57.957308 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-96a8bb55-3b33-4291-9556-d4243b039136 None None] MFA Rules not processed for user `5150c06020594efe957359f641eba5da`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:57.961873 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-96a8bb55-3b33-4291-9556-d4243b039136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:58.121793 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ebab0-f262-4cfb-94aa-c9b687491f2b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:58.141646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f9f47-fbe0-4bbb-830a-1a1007ffa406 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6720518c03413aa8f85e4d6142fe87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:58.144414 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f9f47-fbe0-4bbb-830a-1a1007ffa406 service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:58.145382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f9f47-fbe0-4bbb-830a-1a1007ffa406 service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:58.148629 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f9f47-fbe0-4bbb-830a-1a1007ffa406 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:58.157374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a20f9f47-fbe0-4bbb-830a-1a1007ffa406 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:58.188523 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:58.205475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f9f47-fbe0-4bbb-830a-1a1007ffa406 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:58.211297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f9f47-fbe0-4bbb-830a-1a1007ffa406 service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:58.249744 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 711/1400] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:58.249744 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382748e-1fdc-455b-a24d-bb2c3d6380ff None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:58.249744 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382748e-1fdc-455b-a24d-bb2c3d6380ff None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:58.249744 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382748e-1fdc-455b-a24d-bb2c3d6380ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:58.272034 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 690/1401] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:58.278116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68d57e-98e0-4d2e-a72b-9c9d61d69bdd None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:58.278855 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68d57e-98e0-4d2e-a72b-9c9d61d69bdd None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:58.279528 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68d57e-98e0-4d2e-a72b-9c9d61d69bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:58.318206 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-7382748e-1fdc-455b-a24d-bb2c3d6380ff None None] MFA Rules not processed for user `0b01e277944d4edcb8625ebf4544e340`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:58.320703 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7382748e-1fdc-455b-a24d-bb2c3d6380ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:58.355801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:58.366519 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6e68d57e-98e0-4d2e-a72b-9c9d61d69bdd None None] MFA Rules not processed for user `bc29734859a5460faee189b1726b50b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:58.370735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6e68d57e-98e0-4d2e-a72b-9c9d61d69bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:58.597688 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 691/1402] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:58.614672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68d57e-98e0-4d2e-a72b-9c9d61d69bdd None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:58.618572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 712/1403] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:58.619921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382748e-1fdc-455b-a24d-bb2c3d6380ff None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:58.641737 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6e68d57e-98e0-4d2e-a72b-9c9d61d69bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:58.658721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382748e-1fdc-455b-a24d-bb2c3d6380ff None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:58.677459 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc1749-3cc9-4fb2-9ee9-ca02a1f6a9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:58.678485 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc1749-3cc9-4fb2-9ee9-ca02a1f6a9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:58.678947 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc1749-3cc9-4fb2-9ee9-ca02a1f6a9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:58.679377 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc1749-3cc9-4fb2-9ee9-ca02a1f6a9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:58.680672 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc1749-3cc9-4fb2-9ee9-ca02a1f6a9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:58.688883 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc1749-3cc9-4fb2-9ee9-ca02a1f6a9f9 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:58.710403 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68d57e-98e0-4d2e-a72b-9c9d61d69bdd None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:58.717971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:58.718883 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:58.720219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:58.722108 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] PATH_INFO: `/v3/projects/f2afb4e61d9e4a5bb60826ab3499c8a2` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:58.727175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 713/1404] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:58 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:58.727509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2afb4e61d9e4a5bb60826ab3499c8a2)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:58.730820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:58.734088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc1749-3cc9-4fb2-9ee9-ca02a1f6a9f9 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:58.758459 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc1749-3cc9-4fb2-9ee9-ca02a1f6a9f9 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:58.771919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659a317-1a8e-47d4-8fa2-640e8b9aba6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:58.773759 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659a317-1a8e-47d4-8fa2-640e8b9aba6c admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:58.774031 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659a317-1a8e-47d4-8fa2-640e8b9aba6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:58.774333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659a317-1a8e-47d4-8fa2-640e8b9aba6c admin admin] PATH_INFO: `/v3/projects/cafd5b88d94940b99914be31798f3e57/users/5e76fb7d212d4912b079288aab860d76/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:58.782329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659a317-1a8e-47d4-8fa2-640e8b9aba6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cafd5b88d94940b99914be31798f3e57, user_id=5e76fb7d212d4912b079288aab860d76, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:58.785361 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659a317-1a8e-47d4-8fa2-640e8b9aba6c admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:58.821664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 714/1405] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:58 2022] PUT /identity/v3/projects/cafd5b88d94940b99914be31798f3e57/users/5e76fb7d212d4912b079288aab860d76/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:58.827313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec6220-1a65-4e9c-bae3-cd43ab8eb23d None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:58.828798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec6220-1a65-4e9c-bae3-cd43ab8eb23d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:58.829110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec6220-1a65-4e9c-bae3-cd43ab8eb23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:58.880973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2afb4e61d9e4a5bb60826ab3499c8a2', 'request_id': 'req-280ff843-0673-4f83-ba5e-221072d30502'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:58.908617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2afb4e61d9e4a5bb60826ab3499c8a2', 'request_id': 'req-280ff843-0673-4f83-ba5e-221072d30502'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:58.927913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-7cec6220-1a65-4e9c-bae3-cd43ab8eb23d None None] MFA Rules not processed for user `bc29734859a5460faee189b1726b50b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:58.931085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7cec6220-1a65-4e9c-bae3-cd43ab8eb23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:58.944167 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2afb4e61d9e4a5bb60826ab3499c8a2', 'request_id': 'req-280ff843-0673-4f83-ba5e-221072d30502'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:58.984015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] The token cache is being invalidate because project f2afb4e61d9e4a5bb60826ab3499c8a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:48:58.984424 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:58.988504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:48:58.994824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 692/1406] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:48:58 2022] DELETE /identity/v3/projects/f2afb4e61d9e4a5bb60826ab3499c8a2 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:48:58.998519 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:59.003337 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:59.204126 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 715/1407] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:59.204816 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec6220-1a65-4e9c-bae3-cd43ab8eb23d None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:59.219039 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7cec6220-1a65-4e9c-bae3-cd43ab8eb23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:59.298343 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:59.320270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ff843-0673-4f83-ba5e-221072d30502 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:59.348142 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e281204-7054-41fd-b9b5-c0adb812c027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:59.349396 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e281204-7054-41fd-b9b5-c0adb812c027 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:59.350708 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e281204-7054-41fd-b9b5-c0adb812c027 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:59.350708 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e281204-7054-41fd-b9b5-c0adb812c027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:59.361119 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5e281204-7054-41fd-b9b5-c0adb812c027 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:59.443088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec6220-1a65-4e9c-bae3-cd43ab8eb23d None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:59.468430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92bd3f-7b86-4984-9f0a-3d1068e122ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:59.469210 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92bd3f-7b86-4984-9f0a-3d1068e122ad admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:59.469494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92bd3f-7b86-4984-9f0a-3d1068e122ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:59.469771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92bd3f-7b86-4984-9f0a-3d1068e122ad admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:59.470471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92bd3f-7b86-4984-9f0a-3d1068e122ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:59.478693 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92bd3f-7b86-4984-9f0a-3d1068e122ad admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:59.517037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 716/1408] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:48:59.521309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4df2f1-b82f-465d-bae5-3dd2b37a7cb2 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:59.521309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4df2f1-b82f-465d-bae5-3dd2b37a7cb2 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:59.521309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4df2f1-b82f-465d-bae5-3dd2b37a7cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:59.598929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e281204-7054-41fd-b9b5-c0adb812c027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:59.601116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e281204-7054-41fd-b9b5-c0adb812c027 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:59.637105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 693/1409] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:48:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:48:59.638956 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e281204-7054-41fd-b9b5-c0adb812c027 service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:48:59.646750 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5e281204-7054-41fd-b9b5-c0adb812c027 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:59.676061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-8d4df2f1-b82f-465d-bae5-3dd2b37a7cb2 None None] MFA Rules not processed for user `5e76fb7d212d4912b079288aab860d76`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:48:59.690529 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8d4df2f1-b82f-465d-bae5-3dd2b37a7cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:48:59.756673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e281204-7054-41fd-b9b5-c0adb812c027 service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:48:59.805336 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9910f-313f-434a-a093-201e9e039408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:48:59.812272 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9910f-313f-434a-a093-201e9e039408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:48:59.823910 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9910f-313f-434a-a093-201e9e039408 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:48:59.823910 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9910f-313f-434a-a093-201e9e039408 admin admin] PATH_INFO: `/v3/users/e3f000375aa24b209f9f89c30782bf3b` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:48:59.824310 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:48:59.832028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9910f-313f-434a-a093-201e9e039408 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3f000375aa24b209f9f89c30782bf3b)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:48:59.848229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9910f-313f-434a-a093-201e9e039408 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:48:59.982234 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 717/1410] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:48:59 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:48:59.987031 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4df2f1-b82f-465d-bae5-3dd2b37a7cb2 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:00.008194 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4df2f1-b82f-465d-bae5-3dd2b37a7cb2 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:00.017989 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43462c43-1c2d-4d0e-b87e-11a5df5f22f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:00.018966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43462c43-1c2d-4d0e-b87e-11a5df5f22f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:00.019329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43462c43-1c2d-4d0e-b87e-11a5df5f22f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:00.019668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43462c43-1c2d-4d0e-b87e-11a5df5f22f8 admin admin] PATH_INFO: `/v3/projects/d01bf2944a9e43398b511ab79908e211/users/49329e5ee62f47e496585ee87871d874/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:00.023159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8bc9910f-313f-434a-a093-201e9e039408 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3f000375aa24b209f9f89c30782bf3b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:00.023977 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8bc9910f-313f-434a-a093-201e9e039408 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3f000375aa24b209f9f89c30782bf3b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:00.026670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8bc9910f-313f-434a-a093-201e9e039408 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3f000375aa24b209f9f89c30782bf3b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:00.051532 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43462c43-1c2d-4d0e-b87e-11a5df5f22f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d01bf2944a9e43398b511ab79908e211, user_id=49329e5ee62f47e496585ee87871d874, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:00.056140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43462c43-1c2d-4d0e-b87e-11a5df5f22f8 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:00.056877 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8bc9910f-313f-434a-a093-201e9e039408 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3f000375aa24b209f9f89c30782bf3b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:00.075426 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8bc9910f-313f-434a-a093-201e9e039408 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3f000375aa24b209f9f89c30782bf3b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:00.109664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 718/1411] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:48:59 2022] PUT /identity/v3/projects/d01bf2944a9e43398b511ab79908e211/users/49329e5ee62f47e496585ee87871d874/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:00.109664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43462c43-1c2d-4d0e-b87e-11a5df5f22f8 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:00.110246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 694/1412] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:48:59 2022] DELETE /identity/v3/users/e3f000375aa24b209f9f89c30782bf3b => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:00.110246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9910f-313f-434a-a093-201e9e039408 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:00.111530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8bc9910f-313f-434a-a093-201e9e039408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:00.119062 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-43462c43-1c2d-4d0e-b87e-11a5df5f22f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:00.289906 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9910f-313f-434a-a093-201e9e039408 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:00.301740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8aff6-1146-4128-b750-e6ea3853901d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:00.302625 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8aff6-1146-4128-b750-e6ea3853901d service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:00.302625 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8aff6-1146-4128-b750-e6ea3853901d service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:00.303014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8aff6-1146-4128-b750-e6ea3853901d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:00.310988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43462c43-1c2d-4d0e-b87e-11a5df5f22f8 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:00.311240 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f0e8aff6-1146-4128-b750-e6ea3853901d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:00.415648 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:00.493860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8aff6-1146-4128-b750-e6ea3853901d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:00.495367 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8aff6-1146-4128-b750-e6ea3853901d service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:00.520495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 695/1413] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:00 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:00.522126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a756e8-00f6-4b1c-820a-4ed3b121c098 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:00.522403 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a756e8-00f6-4b1c-820a-4ed3b121c098 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:00.522839 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a756e8-00f6-4b1c-820a-4ed3b121c098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:00.603040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-78a756e8-00f6-4b1c-820a-4ed3b121c098 None None] MFA Rules not processed for user `49329e5ee62f47e496585ee87871d874`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:00.604338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-78a756e8-00f6-4b1c-820a-4ed3b121c098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:00.616188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be633f-a31b-4021-bd42-14264cc12eba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:00.620014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be633f-a31b-4021-bd42-14264cc12eba service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:00.620378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be633f-a31b-4021-bd42-14264cc12eba service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:00.620655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be633f-a31b-4021-bd42-14264cc12eba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:00.636196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-15be633f-a31b-4021-bd42-14264cc12eba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:00.742667 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:00.866201 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be633f-a31b-4021-bd42-14264cc12eba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:00.868383 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be633f-a31b-4021-bd42-14264cc12eba service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:00.919848 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 696/1414] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:00.926936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a756e8-00f6-4b1c-820a-4ed3b121c098 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:00.930642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-78a756e8-00f6-4b1c-820a-4ed3b121c098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:00.945794 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 719/1415] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 845 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:01.006306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be633f-a31b-4021-bd42-14264cc12eba service nova] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:01.008540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-15be633f-a31b-4021-bd42-14264cc12eba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:01.290737 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a756e8-00f6-4b1c-820a-4ed3b121c098 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:01.334851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857b5fd-b060-4943-a436-151c18b14b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:01.336474 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857b5fd-b060-4943-a436-151c18b14b16 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:01.336941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857b5fd-b060-4943-a436-151c18b14b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:01.337402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857b5fd-b060-4943-a436-151c18b14b16 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:01.358830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857b5fd-b060-4943-a436-151c18b14b16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:01.374012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857b5fd-b060-4943-a436-151c18b14b16 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:01.429858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be633f-a31b-4021-bd42-14264cc12eba service nova] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:01.466782 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ac91b-0086-414c-9a8f-346312590535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:01.467962 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ac91b-0086-414c-9a8f-346312590535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:01.468468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ac91b-0086-414c-9a8f-346312590535 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:01.468979 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ac91b-0086-414c-9a8f-346312590535 admin admin] PATH_INFO: `/v3/users/9c1eb56352494cff86f71d80a012d060` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:01.483719 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ac91b-0086-414c-9a8f-346312590535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c1eb56352494cff86f71d80a012d060)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:01.489967 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ac91b-0086-414c-9a8f-346312590535 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:01.543151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 697/1416] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:00 2022] POST /identity/v3/projects => generated 365 bytes in 617 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:01.550936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857b5fd-b060-4943-a436-151c18b14b16 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:01.728489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:01.738608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-245ac91b-0086-414c-9a8f-346312590535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c1eb56352494cff86f71d80a012d060'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:01.775666 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-245ac91b-0086-414c-9a8f-346312590535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c1eb56352494cff86f71d80a012d060'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:01.779653 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-245ac91b-0086-414c-9a8f-346312590535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c1eb56352494cff86f71d80a012d060'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:01.780846 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-245ac91b-0086-414c-9a8f-346312590535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c1eb56352494cff86f71d80a012d060'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:01.807950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-245ac91b-0086-414c-9a8f-346312590535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c1eb56352494cff86f71d80a012d060'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:01.822555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857b5fd-b060-4943-a436-151c18b14b16 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:01.873664 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724d0c2-6ae9-4492-8117-cd062c6dd274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:01.875829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724d0c2-6ae9-4492-8117-cd062c6dd274 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:01.875829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724d0c2-6ae9-4492-8117-cd062c6dd274 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:01.877460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724d0c2-6ae9-4492-8117-cd062c6dd274 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:01.878407 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724d0c2-6ae9-4492-8117-cd062c6dd274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:01.898822 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724d0c2-6ae9-4492-8117-cd062c6dd274 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:01.912530 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 720/1417] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:49:01 2022] DELETE /identity/v3/users/9c1eb56352494cff86f71d80a012d060 => generated 0 bytes in 908 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:01.969012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ac91b-0086-414c-9a8f-346312590535 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:01.972990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-245ac91b-0086-414c-9a8f-346312590535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:02.058275 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 698/1418] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:01 2022] POST /identity/v3/users => generated 289 bytes in 508 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:02.061903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd046bb6-ffdf-45c8-b6cc-2231800238c0 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:02.065031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd046bb6-ffdf-45c8-b6cc-2231800238c0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:02.066756 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd046bb6-ffdf-45c8-b6cc-2231800238c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:02.148890 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:02.191252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ac91b-0086-414c-9a8f-346312590535 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:02.202671 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-cd046bb6-ffdf-45c8-b6cc-2231800238c0 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:02.204825 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cd046bb6-ffdf-45c8-b6cc-2231800238c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:02.206046 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d4366-b028-41af-aa0c-02b70da77acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:02.206870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d4366-b028-41af-aa0c-02b70da77acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:02.209129 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d4366-b028-41af-aa0c-02b70da77acc admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:02.209564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d4366-b028-41af-aa0c-02b70da77acc admin admin] PATH_INFO: `/v3/users/1a23a7ac7dbb469b87a326337d63ae7c` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:02.215043 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d4366-b028-41af-aa0c-02b70da77acc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a23a7ac7dbb469b87a326337d63ae7c)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:02.221697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d4366-b028-41af-aa0c-02b70da77acc admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:02.270019 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 699/1419] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:02.271630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd046bb6-ffdf-45c8-b6cc-2231800238c0 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:02.293350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cd046bb6-ffdf-45c8-b6cc-2231800238c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:02.354855 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd046bb6-ffdf-45c8-b6cc-2231800238c0 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:02.369145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cc410-1f97-481e-b2fa-442d470eb922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:02.370158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cc410-1f97-481e-b2fa-442d470eb922 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:02.370604 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cc410-1f97-481e-b2fa-442d470eb922 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:02.371027 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cc410-1f97-481e-b2fa-442d470eb922 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:02.371852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cc410-1f97-481e-b2fa-442d470eb922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:02.375472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cc410-1f97-481e-b2fa-442d470eb922 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:02.426491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 700/1420] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:02.430777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a94e2-58b5-4d95-a44f-ee28f95af49b None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:02.435834 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a94e2-58b5-4d95-a44f-ee28f95af49b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:02.436108 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a94e2-58b5-4d95-a44f-ee28f95af49b None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:02.446475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 701/1421] 10.222.0.36 () {54 vars in 912 bytes} [Sat Oct 15 13:49:02 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:49:02.458627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a94e2-58b5-4d95-a44f-ee28f95af49b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:02.464156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3c0a94e2-58b5-4d95-a44f-ee28f95af49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:02.479284 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-655d4366-b028-41af-aa0c-02b70da77acc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a23a7ac7dbb469b87a326337d63ae7c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:02.532350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-655d4366-b028-41af-aa0c-02b70da77acc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a23a7ac7dbb469b87a326337d63ae7c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:02.538331 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-655d4366-b028-41af-aa0c-02b70da77acc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a23a7ac7dbb469b87a326337d63ae7c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:02.539642 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-655d4366-b028-41af-aa0c-02b70da77acc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a23a7ac7dbb469b87a326337d63ae7c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:02.568348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-655d4366-b028-41af-aa0c-02b70da77acc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a23a7ac7dbb469b87a326337d63ae7c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:02.687809 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a94e2-58b5-4d95-a44f-ee28f95af49b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:02.692784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555cdbb-5c76-4f5f-99da-ade13df94123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:02.693852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555cdbb-5c76-4f5f-99da-ade13df94123 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:02.694173 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555cdbb-5c76-4f5f-99da-ade13df94123 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:02.694447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555cdbb-5c76-4f5f-99da-ade13df94123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:02.697478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f555cdbb-5c76-4f5f-99da-ade13df94123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:02.699514 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 721/1422] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:49:01 2022] DELETE /identity/v3/users/1a23a7ac7dbb469b87a326337d63ae7c => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:02.711117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d4366-b028-41af-aa0c-02b70da77acc admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:02.717629 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-655d4366-b028-41af-aa0c-02b70da77acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:02.736617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555cdbb-5c76-4f5f-99da-ade13df94123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:02.738185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555cdbb-5c76-4f5f-99da-ade13df94123 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:02.760263 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 702/1423] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:02.761758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaca3a-da70-4d29-870e-20fa006869dc None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:02.762051 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaca3a-da70-4d29-870e-20fa006869dc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:02.762383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaca3a-da70-4d29-870e-20fa006869dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:02.817191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5cdaca3a-da70-4d29-870e-20fa006869dc None None] MFA Rules not processed for user `90985ec8c43a4d7895942731dc1d002c`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:02.819247 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5cdaca3a-da70-4d29-870e-20fa006869dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:02.972068 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 703/1424] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:49:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:02.973948 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf9afe-45c4-4b23-8b47-99940422aa89 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:02.974273 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf9afe-45c4-4b23-8b47-99940422aa89 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:02.974443 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf9afe-45c4-4b23-8b47-99940422aa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:02.981845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d4366-b028-41af-aa0c-02b70da77acc admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:02.997823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc6661-df52-4683-a0e7-d6cc7b479758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:02.998867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc6661-df52-4683-a0e7-d6cc7b479758 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:02.999972 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc6661-df52-4683-a0e7-d6cc7b479758 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:03.000593 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc6661-df52-4683-a0e7-d6cc7b479758 admin admin] PATH_INFO: `/v3/projects/59bbc598003c4f5286374b105c2e592a/users/854d3e695b354a17989607f3153d8c6b/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:03.035226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5fcf9afe-45c4-4b23-8b47-99940422aa89 None None] MFA Rules not processed for user `583cce1547144e11978f20a3193ceba4`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:03.036582 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5fcf9afe-45c4-4b23-8b47-99940422aa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:03.048231 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc6661-df52-4683-a0e7-d6cc7b479758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bbc598003c4f5286374b105c2e592a, user_id=854d3e695b354a17989607f3153d8c6b, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:03.063902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc6661-df52-4683-a0e7-d6cc7b479758 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:03.106984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 722/1425] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:02 2022] PUT /identity/v3/projects/59bbc598003c4f5286374b105c2e592a/users/854d3e695b354a17989607f3153d8c6b/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:03.124086 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ba052-c795-48ab-bc75-a4c93ad8a27a None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:03.126901 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ba052-c795-48ab-bc75-a4c93ad8a27a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:03.126901 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ba052-c795-48ab-bc75-a4c93ad8a27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:03.185518 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-632ba052-c795-48ab-bc75-a4c93ad8a27a None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:03.190307 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-632ba052-c795-48ab-bc75-a4c93ad8a27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:03.323124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:03.372871 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 723/1426] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:03.375244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ba052-c795-48ab-bc75-a4c93ad8a27a None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:03.379445 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-632ba052-c795-48ab-bc75-a4c93ad8a27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:03.429297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 704/1427] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:03.438408 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf9afe-45c4-4b23-8b47-99940422aa89 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:03.447204 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5fcf9afe-45c4-4b23-8b47-99940422aa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:03.550100 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf9afe-45c4-4b23-8b47-99940422aa89 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:03.581029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:03.585157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:03.586077 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:03.586886 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] PATH_INFO: `/v3/projects/dabf57723d854192a1782ad10b212e2e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:03.592361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dabf57723d854192a1782ad10b212e2e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:03.598565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:03.617888 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:03.701301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ba052-c795-48ab-bc75-a4c93ad8a27a None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:03.709809 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10654-975b-46a5-9585-bde0222e77ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:03.712829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10654-975b-46a5-9585-bde0222e77ca service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:03.714684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10654-975b-46a5-9585-bde0222e77ca service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:03.715176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10654-975b-46a5-9585-bde0222e77ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:03.739205 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10654-975b-46a5-9585-bde0222e77ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:03.742871 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10654-975b-46a5-9585-bde0222e77ca service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:03.780211 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 724/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:03.782254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10654-975b-46a5-9585-bde0222e77ca service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:03.873426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10654-975b-46a5-9585-bde0222e77ca service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:03.902635 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d16ee-38e6-4288-9e10-b8683ea40665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:03.906628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d16ee-38e6-4288-9e10-b8683ea40665 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:03.908637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d16ee-38e6-4288-9e10-b8683ea40665 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:03.910694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d16ee-38e6-4288-9e10-b8683ea40665 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:03.913734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d16ee-38e6-4288-9e10-b8683ea40665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:03.934895 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d16ee-38e6-4288-9e10-b8683ea40665 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:03.943249 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dabf57723d854192a1782ad10b212e2e', 'request_id': 'req-77d88383-89db-4b4a-a063-5b18097b1eb3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:03.990953 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 725/1429] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:04.001033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dabf57723d854192a1782ad10b212e2e', 'request_id': 'req-77d88383-89db-4b4a-a063-5b18097b1eb3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:04.013965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d16ee-38e6-4288-9e10-b8683ea40665 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:04.049218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dabf57723d854192a1782ad10b212e2e', 'request_id': 'req-77d88383-89db-4b4a-a063-5b18097b1eb3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:04.076791 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d16ee-38e6-4288-9e10-b8683ea40665 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:04.100701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] The token cache is being invalidate because project dabf57723d854192a1782ad10b212e2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:49:04.101229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:04.106283 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:04.120368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 705/1430] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:49:03 2022] DELETE /identity/v3/projects/dabf57723d854192a1782ad10b212e2e => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:04.122145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:04.128663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:04.343412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d88383-89db-4b4a-a063-5b18097b1eb3 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:04.359172 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15094653-0bc6-4fe2-811f-ceb8f77586c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:04.359912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15094653-0bc6-4fe2-811f-ceb8f77586c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:04.360200 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15094653-0bc6-4fe2-811f-ceb8f77586c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:04.360627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15094653-0bc6-4fe2-811f-ceb8f77586c6 admin admin] PATH_INFO: `/v3/projects/59bbc598003c4f5286374b105c2e592a/users/854d3e695b354a17989607f3153d8c6b/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:04.367432 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15094653-0bc6-4fe2-811f-ceb8f77586c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bbc598003c4f5286374b105c2e592a, user_id=854d3e695b354a17989607f3153d8c6b, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:04.373546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15094653-0bc6-4fe2-811f-ceb8f77586c6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:04.395803 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12f635-35c2-4f0e-b03e-57c8ba11f651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:04.397008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12f635-35c2-4f0e-b03e-57c8ba11f651 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:04.397492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12f635-35c2-4f0e-b03e-57c8ba11f651 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:04.397990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12f635-35c2-4f0e-b03e-57c8ba11f651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:04.416069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-cf12f635-35c2-4f0e-b03e-57c8ba11f651 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:04.437943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 706/1431] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:04 2022] PUT /identity/v3/projects/59bbc598003c4f5286374b105c2e592a/users/854d3e695b354a17989607f3153d8c6b/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:04.440456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072b145-f62e-41f6-ac3c-81bed13d0981 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:04.440940 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072b145-f62e-41f6-ac3c-81bed13d0981 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:04.441412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072b145-f62e-41f6-ac3c-81bed13d0981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:04.459927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12f635-35c2-4f0e-b03e-57c8ba11f651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:04.461468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12f635-35c2-4f0e-b03e-57c8ba11f651 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:04.504845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 726/1432] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:04.506473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e13ee-fb82-4746-bd93-b7e3ee8fa2ae None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:04.506766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e13ee-fb82-4746-bd93-b7e3ee8fa2ae None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:04.507034 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e13ee-fb82-4746-bd93-b7e3ee8fa2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:04.522809 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9072b145-f62e-41f6-ac3c-81bed13d0981 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:04.527401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9072b145-f62e-41f6-ac3c-81bed13d0981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:04.634565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-136e13ee-fb82-4746-bd93-b7e3ee8fa2ae None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:04.639404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-136e13ee-fb82-4746-bd93-b7e3ee8fa2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:04.778389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:04.794235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 727/1433] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:49:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:04.796037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93340b69-80e6-4348-973a-4330b6693d25 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:04.797166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93340b69-80e6-4348-973a-4330b6693d25 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:04.797166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93340b69-80e6-4348-973a-4330b6693d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:04.809012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 707/1434] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:04.827940 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cbadf-65f0-438e-9692-be8c438ceff4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:04.828434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cbadf-65f0-438e-9692-be8c438ceff4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:04.828929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cbadf-65f0-438e-9692-be8c438ceff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:04.903131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-93340b69-80e6-4348-973a-4330b6693d25 None None] MFA Rules not processed for user `854d3e695b354a17989607f3153d8c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:04.904727 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-93340b69-80e6-4348-973a-4330b6693d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:04.933786 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ab7cbadf-65f0-438e-9692-be8c438ceff4 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:04.935238 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ab7cbadf-65f0-438e-9692-be8c438ceff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:04.964763 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 708/1435] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:04.966101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cbadf-65f0-438e-9692-be8c438ceff4 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:04.993179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cbadf-65f0-438e-9692-be8c438ceff4 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:05.003267 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf230e0-11c6-4c0b-bd85-c41cd474d120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:05.003947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf230e0-11c6-4c0b-bd85-c41cd474d120 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:05.004220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf230e0-11c6-4c0b-bd85-c41cd474d120 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:05.005098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf230e0-11c6-4c0b-bd85-c41cd474d120 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:05.006083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf230e0-11c6-4c0b-bd85-c41cd474d120 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:49:05.006561 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf230e0-11c6-4c0b-bd85-c41cd474d120 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:05.008645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf230e0-11c6-4c0b-bd85-c41cd474d120 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:05.032570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 709/1436] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:49:05.039048 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf230e0-11c6-4c0b-bd85-c41cd474d120 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:05.084832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf230e0-11c6-4c0b-bd85-c41cd474d120 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:05.109684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e1093-675e-4013-9419-4b0303fecd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:05.114011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e1093-675e-4013-9419-4b0303fecd81 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:05.118342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e1093-675e-4013-9419-4b0303fecd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:05.119819 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e1093-675e-4013-9419-4b0303fecd81 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:05.124896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e1093-675e-4013-9419-4b0303fecd81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:05.138704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e1093-675e-4013-9419-4b0303fecd81 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:05.160304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 728/1437] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:05.160906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93340b69-80e6-4348-973a-4330b6693d25 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:05.166204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-93340b69-80e6-4348-973a-4330b6693d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:05.167646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:05.264419 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 710/1438] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:05 2022] POST /identity/v3/projects => generated 373 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:05.267534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e1093-675e-4013-9419-4b0303fecd81 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:05.276281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-899e1093-675e-4013-9419-4b0303fecd81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:05.530961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93340b69-80e6-4348-973a-4330b6693d25 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:05.546985 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e1093-675e-4013-9419-4b0303fecd81 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:05.577669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b544d-9481-43ad-8e36-0b1bf3f42612 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:05.585265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b544d-9481-43ad-8e36-0b1bf3f42612 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:05.585804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b544d-9481-43ad-8e36-0b1bf3f42612 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:05.586209 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b544d-9481-43ad-8e36-0b1bf3f42612 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:05.701967 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e5713-22a5-455d-9fce-8401ee503a64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:05.702634 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e5713-22a5-455d-9fce-8401ee503a64 service swift] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:05.702920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e5713-22a5-455d-9fce-8401ee503a64 service swift] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:05.703193 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e5713-22a5-455d-9fce-8401ee503a64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:05.716264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:05.851776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b544d-9481-43ad-8e36-0b1bf3f42612 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:05.858085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b544d-9481-43ad-8e36-0b1bf3f42612 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:05.889447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e5713-22a5-455d-9fce-8401ee503a64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:05.891165 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e5713-22a5-455d-9fce-8401ee503a64 service swift] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:05.895610 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 711/1439] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:05.898331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b544d-9481-43ad-8e36-0b1bf3f42612 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:05.935246 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 729/1440] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:49:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 777 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:49:06.151996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e5713-22a5-455d-9fce-8401ee503a64 service swift] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:06.152929 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-047e5713-22a5-455d-9fce-8401ee503a64 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:06.258130 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b544d-9481-43ad-8e36-0b1bf3f42612 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:06.288406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a0a60-1d75-4326-b0ad-97929c1ff62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:06.291477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a0a60-1d75-4326-b0ad-97929c1ff62e admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:06.293558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a0a60-1d75-4326-b0ad-97929c1ff62e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:06.300242 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a0a60-1d75-4326-b0ad-97929c1ff62e admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:06.301705 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a0a60-1d75-4326-b0ad-97929c1ff62e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:06.315787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a0a60-1d75-4326-b0ad-97929c1ff62e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:06.428678 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 712/1441] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:05 2022] POST /identity/v3/users => generated 287 bytes in 527 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:06.430844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a0a60-1d75-4326-b0ad-97929c1ff62e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:06.488092 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a0a60-1d75-4326-b0ad-97929c1ff62e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:06.517503 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319d463-f6ca-4e65-be37-50413949c781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:06.522669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319d463-f6ca-4e65-be37-50413949c781 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:06.523239 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319d463-f6ca-4e65-be37-50413949c781 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:06.523800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319d463-f6ca-4e65-be37-50413949c781 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:06.525200 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319d463-f6ca-4e65-be37-50413949c781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:06.528584 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319d463-f6ca-4e65-be37-50413949c781 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:06.538494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e5713-22a5-455d-9fce-8401ee503a64 service swift] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:06.574004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:06.575072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:06.576077 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:06.576653 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] PATH_INFO: `/v3/projects/048e14a2201e407b92e80c8a758159b7` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:06.594775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=048e14a2201e407b92e80c8a758159b7)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:06.598654 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:06.609701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 713/1442] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:06.616961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319d463-f6ca-4e65-be37-50413949c781 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:06.661060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319d463-f6ca-4e65-be37-50413949c781 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:06.701441 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a35fb-75dc-493b-8d55-04170441ac30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:06.702634 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a35fb-75dc-493b-8d55-04170441ac30 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:06.703147 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a35fb-75dc-493b-8d55-04170441ac30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:06.704518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a35fb-75dc-493b-8d55-04170441ac30 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:06.705764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a35fb-75dc-493b-8d55-04170441ac30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:06.732775 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a35fb-75dc-493b-8d55-04170441ac30 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:06.786460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 714/1443] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:49:06.788243 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a35fb-75dc-493b-8d55-04170441ac30 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:06.822437 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:06.851592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a35fb-75dc-493b-8d55-04170441ac30 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:06.864407 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '048e14a2201e407b92e80c8a758159b7', 'request_id': 'req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:06.900821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '048e14a2201e407b92e80c8a758159b7', 'request_id': 'req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:06.941652 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '048e14a2201e407b92e80c8a758159b7', 'request_id': 'req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:07.012007 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] The token cache is being invalidate because project 048e14a2201e407b92e80c8a758159b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:49:07.012531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:07.029940 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:07.040538 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 730/1444] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:49:06 2022] DELETE /identity/v3/projects/048e14a2201e407b92e80c8a758159b7 => generated 0 bytes in 886 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:49:07.050762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:07.060326 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:07.188614 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9894ed-5b5b-4deb-965e-1ed11b23824e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:07.189666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9894ed-5b5b-4deb-965e-1ed11b23824e admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:07.190071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9894ed-5b5b-4deb-965e-1ed11b23824e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:07.190596 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9894ed-5b5b-4deb-965e-1ed11b23824e admin admin] PATH_INFO: `/v3/projects/3d590755103141aaa003054cb889a183/users/10364e42b429400fb014eb5f6d60e3bb/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:07.257095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9894ed-5b5b-4deb-965e-1ed11b23824e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d590755103141aaa003054cb889a183, user_id=10364e42b429400fb014eb5f6d60e3bb, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:07.280901 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9894ed-5b5b-4deb-965e-1ed11b23824e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:07.294961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:07.314413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 715/1445] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:06 2022] PUT /identity/v3/projects/3d590755103141aaa003054cb889a183/users/10364e42b429400fb014eb5f6d60e3bb/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:07.316067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9894ed-5b5b-4deb-965e-1ed11b23824e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:07.326434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9b9894ed-5b5b-4deb-965e-1ed11b23824e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:07.473285 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fa34e-e1e7-48bf-a1b5-e520152e1fc2 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:07.815941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9894ed-5b5b-4deb-965e-1ed11b23824e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:07.835388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806bc019-94b8-4da9-b91f-f6c80ce83ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:07.836791 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bc019-94b8-4da9-b91f-f6c80ce83ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:07.837514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bc019-94b8-4da9-b91f-f6c80ce83ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:07.838146 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806bc019-94b8-4da9-b91f-f6c80ce83ffe admin admin] PATH_INFO: `/v3/users/5e76fb7d212d4912b079288aab860d76` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:07.842513 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bc019-94b8-4da9-b91f-f6c80ce83ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e76fb7d212d4912b079288aab860d76)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:07.867436 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806bc019-94b8-4da9-b91f-f6c80ce83ffe admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:07.911935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:07.914246 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:07.914246 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:07.919800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] PATH_INFO: `/v3/projects/f8b834400955461d9dbe38f92950eb1e` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:07.940766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8b834400955461d9dbe38f92950eb1e)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:07.993204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:08.154399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-806bc019-94b8-4da9-b91f-f6c80ce83ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e76fb7d212d4912b079288aab860d76'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:08.156282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-806bc019-94b8-4da9-b91f-f6c80ce83ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e76fb7d212d4912b079288aab860d76'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:08.158051 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-806bc019-94b8-4da9-b91f-f6c80ce83ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e76fb7d212d4912b079288aab860d76'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:08.230720 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-806bc019-94b8-4da9-b91f-f6c80ce83ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e76fb7d212d4912b079288aab860d76'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:08.266308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-806bc019-94b8-4da9-b91f-f6c80ce83ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e76fb7d212d4912b079288aab860d76'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:08.410116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 716/1446] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:49:07 2022] DELETE /identity/v3/users/5e76fb7d212d4912b079288aab860d76 => generated 0 bytes in 1095 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:08.430136 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219f464-65b2-44a5-9c7b-0030b0433bc4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:08.436861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219f464-65b2-44a5-9c7b-0030b0433bc4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:08.437442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b834400955461d9dbe38f92950eb1e', 'request_id': 'req-87e6ef76-5966-4158-b3e8-e03c56291b94'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:08.438827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219f464-65b2-44a5-9c7b-0030b0433bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:08.492775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b834400955461d9dbe38f92950eb1e', 'request_id': 'req-87e6ef76-5966-4158-b3e8-e03c56291b94'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:08.521120 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:08.559091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b834400955461d9dbe38f92950eb1e', 'request_id': 'req-87e6ef76-5966-4158-b3e8-e03c56291b94'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:08.644679 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] The token cache is being invalidate because project f8b834400955461d9dbe38f92950eb1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:49:08.645176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:08.658201 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:08.664397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d219f464-65b2-44a5-9c7b-0030b0433bc4 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:08.666810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d219f464-65b2-44a5-9c7b-0030b0433bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:08.679134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 731/1447] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:49:07 2022] DELETE /identity/v3/projects/f8b834400955461d9dbe38f92950eb1e => generated 0 bytes in 1633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:08.681061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:08.686142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:09.057577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 717/1448] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 640 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:09.059430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219f464-65b2-44a5-9c7b-0030b0433bc4 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:09.072506 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d219f464-65b2-44a5-9c7b-0030b0433bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:09.156787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219f464-65b2-44a5-9c7b-0030b0433bc4 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:09.176109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6ef76-5966-4158-b3e8-e03c56291b94 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:09.189197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10217782-2146-400d-ab67-bc2ed3462df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:09.192245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10217782-2146-400d-ab67-bc2ed3462df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:09.193845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10217782-2146-400d-ab67-bc2ed3462df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:09.195667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10217782-2146-400d-ab67-bc2ed3462df5 admin admin] PATH_INFO: `/v3/users/7f3cb71962c84777bff5f8c651d85300` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:09.197996 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10217782-2146-400d-ab67-bc2ed3462df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f3cb71962c84777bff5f8c651d85300)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:09.204933 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e4d50-777d-4456-bc66-b48e84a2a0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:09.205526 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10217782-2146-400d-ab67-bc2ed3462df5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:09.206185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e4d50-777d-4456-bc66-b48e84a2a0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:09.206686 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e4d50-777d-4456-bc66-b48e84a2a0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:09.207176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e4d50-777d-4456-bc66-b48e84a2a0b2 admin admin] PATH_INFO: `/v3/users/49329e5ee62f47e496585ee87871d874` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:09.217250 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e4d50-777d-4456-bc66-b48e84a2a0b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49329e5ee62f47e496585ee87871d874)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:09.231523 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e4d50-777d-4456-bc66-b48e84a2a0b2 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:09.323712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:09.413249 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-454e4d50-777d-4456-bc66-b48e84a2a0b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49329e5ee62f47e496585ee87871d874'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:09.431649 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-454e4d50-777d-4456-bc66-b48e84a2a0b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49329e5ee62f47e496585ee87871d874'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:09.432921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-454e4d50-777d-4456-bc66-b48e84a2a0b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49329e5ee62f47e496585ee87871d874'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:09.433811 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-454e4d50-777d-4456-bc66-b48e84a2a0b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49329e5ee62f47e496585ee87871d874'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:09.480936 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-454e4d50-777d-4456-bc66-b48e84a2a0b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49329e5ee62f47e496585ee87871d874'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:09.511191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-10217782-2146-400d-ab67-bc2ed3462df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f3cb71962c84777bff5f8c651d85300'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:09.515987 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-10217782-2146-400d-ab67-bc2ed3462df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f3cb71962c84777bff5f8c651d85300'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:09.517412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 732/1449] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:49:08 2022] DELETE /identity/v3/users/49329e5ee62f47e496585ee87871d874 => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:09.518726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e4d50-777d-4456-bc66-b48e84a2a0b2 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:09.523687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-10217782-2146-400d-ab67-bc2ed3462df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f3cb71962c84777bff5f8c651d85300'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:09.525113 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-454e4d50-777d-4456-bc66-b48e84a2a0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:09.623282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-10217782-2146-400d-ab67-bc2ed3462df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f3cb71962c84777bff5f8c651d85300'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:09.670064 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-10217782-2146-400d-ab67-bc2ed3462df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f3cb71962c84777bff5f8c651d85300'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:09.686456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e4d50-777d-4456-bc66-b48e84a2a0b2 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:09.694510 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517fd34-f51d-4069-bc95-b90b89c61fef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:09.698756 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517fd34-f51d-4069-bc95-b90b89c61fef service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:09.699204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517fd34-f51d-4069-bc95-b90b89c61fef service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:09.699628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517fd34-f51d-4069-bc95-b90b89c61fef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:09.706293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9517fd34-f51d-4069-bc95-b90b89c61fef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:09.745667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 718/1450] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:49:09 2022] DELETE /identity/v3/users/7f3cb71962c84777bff5f8c651d85300 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:09.745667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4aee3d-de99-49ce-9ac5-4ce0c60cd8f7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:09.763140 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4aee3d-de99-49ce-9ac5-4ce0c60cd8f7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:09.763618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4aee3d-de99-49ce-9ac5-4ce0c60cd8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:09.943427 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6d4aee3d-de99-49ce-9ac5-4ce0c60cd8f7 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:09.962771 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6d4aee3d-de99-49ce-9ac5-4ce0c60cd8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:10.040855 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517fd34-f51d-4069-bc95-b90b89c61fef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:10.057221 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517fd34-f51d-4069-bc95-b90b89c61fef service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:10.112426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 733/1451] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:10.401161 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 719/1452] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 658 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:10.497264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517fd34-f51d-4069-bc95-b90b89c61fef service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:10.506999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9517fd34-f51d-4069-bc95-b90b89c61fef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:10.532697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517fd34-f51d-4069-bc95-b90b89c61fef service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:10.746379 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:10.751554 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a69bd-b7fc-43a5-b5b3-9d5fc1770b56 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:10.752352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a69bd-b7fc-43a5-b5b3-9d5fc1770b56 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:10.752881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a69bd-b7fc-43a5-b5b3-9d5fc1770b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:10.781285 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6246d52-5986-4725-b412-04a8edd728cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:10.781962 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246d52-5986-4725-b412-04a8edd728cd service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:10.782252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246d52-5986-4725-b412-04a8edd728cd service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:10.782536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6246d52-5986-4725-b412-04a8edd728cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:10.831066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6246d52-5986-4725-b412-04a8edd728cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:10.841525 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-4c0a69bd-b7fc-43a5-b5b3-9d5fc1770b56 None None] MFA Rules not processed for user `90985ec8c43a4d7895942731dc1d002c`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:10.846257 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6246d52-5986-4725-b412-04a8edd728cd service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:10.850157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4c0a69bd-b7fc-43a5-b5b3-9d5fc1770b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:10.893640 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 720/1453] 10.222.0.40 () {58 vars in 1015 bytes} [Sat Oct 15 13:49:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:10.913183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 734/1454] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:49:10.955969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6246d52-5986-4725-b412-04a8edd728cd service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:10.956593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a69bd-b7fc-43a5-b5b3-9d5fc1770b56 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:10.959181 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b6246d52-5986-4725-b412-04a8edd728cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:10.961190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4c0a69bd-b7fc-43a5-b5b3-9d5fc1770b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:10.999663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a69bd-b7fc-43a5-b5b3-9d5fc1770b56 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:11.002204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6246d52-5986-4725-b412-04a8edd728cd service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:11.017459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d634f-c76e-442a-8f24-9075ae82a9e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:11.019006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285b945-f1d1-4187-b97f-5b2c67a5ef49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:11.019705 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285b945-f1d1-4187-b97f-5b2c67a5ef49 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:11.024118 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d634f-c76e-442a-8f24-9075ae82a9e3 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:11.024529 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d634f-c76e-442a-8f24-9075ae82a9e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:11.024810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d634f-c76e-442a-8f24-9075ae82a9e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:11.025820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285b945-f1d1-4187-b97f-5b2c67a5ef49 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:11.026140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285b945-f1d1-4187-b97f-5b2c67a5ef49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:11.050828 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d634f-c76e-442a-8f24-9075ae82a9e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:11.051969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285b945-f1d1-4187-b97f-5b2c67a5ef49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:11.053869 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d634f-c76e-442a-8f24-9075ae82a9e3 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:11.055771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5285b945-f1d1-4187-b97f-5b2c67a5ef49 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:11.072254 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:11.095525 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 735/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:11.106180 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 721/1456] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:13.361259 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285b945-f1d1-4187-b97f-5b2c67a5ef49 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:13.364963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-5285b945-f1d1-4187-b97f-5b2c67a5ef49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:13.440452 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285b945-f1d1-4187-b97f-5b2c67a5ef49 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:13.469663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:13.471456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:13.472330 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:13.473118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c admin admin] PATH_INFO: `/v3/projects/cafd5b88d94940b99914be31798f3e57` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:13.478188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cafd5b88d94940b99914be31798f3e57)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:13.489728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:13.702163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cafd5b88d94940b99914be31798f3e57', 'request_id': 'req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:13.724469 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cafd5b88d94940b99914be31798f3e57', 'request_id': 'req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:13.737216 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cafd5b88d94940b99914be31798f3e57', 'request_id': 'req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:13.761021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c admin admin] The token cache is being invalidate because project cafd5b88d94940b99914be31798f3e57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:49:13.761844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:13.762823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6afef18b-797e-4c6c-a760-a9a5cfb8cf9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:13.772450 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f041ffa-5915-48c7-ae37-11daeec13690 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:13.773223 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f041ffa-5915-48c7-ae37-11daeec13690 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:13.773223 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f041ffa-5915-48c7-ae37-11daeec13690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:13.774109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 736/1457] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:49:13 2022] DELETE /identity/v3/projects/cafd5b88d94940b99914be31798f3e57 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:13.814521 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8f041ffa-5915-48c7-ae37-11daeec13690 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:13.817984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8f041ffa-5915-48c7-ae37-11daeec13690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:13.976181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 722/1458] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:49:13.989801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f0e13-e2bd-4293-80fd-8d305c137937 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:13.990098 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f0e13-e2bd-4293-80fd-8d305c137937 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:13.990422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f0e13-e2bd-4293-80fd-8d305c137937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:14.049835 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-245f0e13-e2bd-4293-80fd-8d305c137937 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:14.053140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-245f0e13-e2bd-4293-80fd-8d305c137937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:14.081068 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 737/1459] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:14.087550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f041ffa-5915-48c7-ae37-11daeec13690 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:14.106940 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f041ffa-5915-48c7-ae37-11daeec13690 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:14.119373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db41de-1249-413e-8902-324584e46d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:14.120481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db41de-1249-413e-8902-324584e46d92 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:14.121005 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db41de-1249-413e-8902-324584e46d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:14.121489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db41de-1249-413e-8902-324584e46d92 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:14.122433 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db41de-1249-413e-8902-324584e46d92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:49:14.122981 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db41de-1249-413e-8902-324584e46d92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:14.125004 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db41de-1249-413e-8902-324584e46d92 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:14.157285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 723/1460] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:49:14.163865 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f0e13-e2bd-4293-80fd-8d305c137937 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:14.180013 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f0e13-e2bd-4293-80fd-8d305c137937 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:14.197847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905a42a-bfa2-4330-8682-3fd1169bf53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:14.198574 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905a42a-bfa2-4330-8682-3fd1169bf53b admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:14.198855 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905a42a-bfa2-4330-8682-3fd1169bf53b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:14.201574 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905a42a-bfa2-4330-8682-3fd1169bf53b admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:14.202353 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905a42a-bfa2-4330-8682-3fd1169bf53b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:14.208802 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905a42a-bfa2-4330-8682-3fd1169bf53b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:14.303167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 738/1461] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:14 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:14.309520 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db41de-1249-413e-8902-324584e46d92 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:14.398390 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db41de-1249-413e-8902-324584e46d92 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:14.784283 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e731a4d-761f-40f4-8f6c-3b03d71b377e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:14.785906 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731a4d-761f-40f4-8f6c-3b03d71b377e admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:14.786557 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731a4d-761f-40f4-8f6c-3b03d71b377e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:14.788555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731a4d-761f-40f4-8f6c-3b03d71b377e admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:14.792666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e731a4d-761f-40f4-8f6c-3b03d71b377e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:14.799777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e731a4d-761f-40f4-8f6c-3b03d71b377e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:14.865427 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 724/1462] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:14 2022] POST /identity/v3/users => generated 294 bytes in 556 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:49:14.871351 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905a42a-bfa2-4330-8682-3fd1169bf53b admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:14.920923 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905a42a-bfa2-4330-8682-3fd1169bf53b admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:14.930772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e395451-1afa-44c8-b7a7-23b0d3c0a3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:14.931825 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e395451-1afa-44c8-b7a7-23b0d3c0a3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:14.932418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e395451-1afa-44c8-b7a7-23b0d3c0a3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:14.932798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e395451-1afa-44c8-b7a7-23b0d3c0a3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:14.934363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e395451-1afa-44c8-b7a7-23b0d3c0a3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:14.938006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e395451-1afa-44c8-b7a7-23b0d3c0a3a4 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:14.968137 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e731a4d-761f-40f4-8f6c-3b03d71b377e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:14.971083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 739/1463] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:14.997021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e731a4d-761f-40f4-8f6c-3b03d71b377e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:15.030864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a412f5-41ce-4660-8d2c-5ce8175b3ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:15.031652 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a412f5-41ce-4660-8d2c-5ce8175b3ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:15.032041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a412f5-41ce-4660-8d2c-5ce8175b3ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:15.032464 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a412f5-41ce-4660-8d2c-5ce8175b3ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:15.033421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a412f5-41ce-4660-8d2c-5ce8175b3ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:15.035552 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a412f5-41ce-4660-8d2c-5ce8175b3ca6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:15.063186 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 725/1464] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:49:15.069694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e395451-1afa-44c8-b7a7-23b0d3c0a3a4 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:15.109537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e395451-1afa-44c8-b7a7-23b0d3c0a3a4 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:15.134277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb811f87-8d1f-4a08-b66c-b3607d2a2ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:15.135644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb811f87-8d1f-4a08-b66c-b3607d2a2ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:15.136296 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb811f87-8d1f-4a08-b66c-b3607d2a2ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:15.136843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb811f87-8d1f-4a08-b66c-b3607d2a2ac1 admin admin] PATH_INFO: `/v3/projects/a65e5b026e40466fb3d657d4cfa732b5/users/8467bbcb8ede41b69c67e55b106817c9/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:15.210519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb811f87-8d1f-4a08-b66c-b3607d2a2ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a65e5b026e40466fb3d657d4cfa732b5, user_id=8467bbcb8ede41b69c67e55b106817c9, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:15.232464 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb811f87-8d1f-4a08-b66c-b3607d2a2ac1 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:15.319831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 740/1465] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:15 2022] PUT /identity/v3/projects/a65e5b026e40466fb3d657d4cfa732b5/users/8467bbcb8ede41b69c67e55b106817c9/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:15.341773 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4d54d-6fa9-4f3f-9d6b-3a1b93930b2c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:15.341959 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4d54d-6fa9-4f3f-9d6b-3a1b93930b2c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:15.342286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4d54d-6fa9-4f3f-9d6b-3a1b93930b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:15.455134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-74d4d54d-6fa9-4f3f-9d6b-3a1b93930b2c None None] MFA Rules not processed for user `8467bbcb8ede41b69c67e55b106817c9`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:15.457295 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-74d4d54d-6fa9-4f3f-9d6b-3a1b93930b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:15.691642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:15.740169 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 726/1466] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:15.741118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb811f87-8d1f-4a08-b66c-b3607d2a2ac1 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:15.938128 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4d54d-6fa9-4f3f-9d6b-3a1b93930b2c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:15.941688 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-74d4d54d-6fa9-4f3f-9d6b-3a1b93930b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:15.945490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:15.997696 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb811f87-8d1f-4a08-b66c-b3607d2a2ac1 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:16.025774 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1cf5d-d069-41bf-b919-508abe7f75bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:16.027080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1cf5d-d069-41bf-b919-508abe7f75bd admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:16.031234 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1cf5d-d069-41bf-b919-508abe7f75bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:16.032080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1cf5d-d069-41bf-b919-508abe7f75bd admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:16.033777 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1cf5d-d069-41bf-b919-508abe7f75bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:16.036883 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1cf5d-d069-41bf-b919-508abe7f75bd admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:16.107587 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 741/1467] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:15 2022] POST /identity/v3/projects => generated 385 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:49:16.111207 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1cf5d-d069-41bf-b919-508abe7f75bd admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:16.166263 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4d54d-6fa9-4f3f-9d6b-3a1b93930b2c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:16.369406 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1cf5d-d069-41bf-b919-508abe7f75bd admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:16.399580 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680d94e-f77e-42b7-9339-88178bf57504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:16.400798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680d94e-f77e-42b7-9339-88178bf57504 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:16.401303 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680d94e-f77e-42b7-9339-88178bf57504 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:16.401851 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680d94e-f77e-42b7-9339-88178bf57504 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:16.403026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680d94e-f77e-42b7-9339-88178bf57504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:16.407186 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680d94e-f77e-42b7-9339-88178bf57504 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:16.420186 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:16.421341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:16.421764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:16.422592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] PATH_INFO: `/v3/projects/d01bf2944a9e43398b511ab79908e211` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:16.432846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] RBAC: Authorizing `identity:delete_project(project_id=d01bf2944a9e43398b511ab79908e211)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:16.435647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:16.494267 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 742/1468] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:16 2022] POST /identity/v3/users => generated 299 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:49:16.516408 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680d94e-f77e-42b7-9339-88178bf57504 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:16.520835 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8680d94e-f77e-42b7-9339-88178bf57504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:16.586066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680d94e-f77e-42b7-9339-88178bf57504 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:16.662020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd01bf2944a9e43398b511ab79908e211', 'request_id': 'req-1fe9aefa-d349-4bf6-9052-0198ef50e9af'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:16.676834 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd01bf2944a9e43398b511ab79908e211', 'request_id': 'req-1fe9aefa-d349-4bf6-9052-0198ef50e9af'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:16.690237 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd01bf2944a9e43398b511ab79908e211', 'request_id': 'req-1fe9aefa-d349-4bf6-9052-0198ef50e9af'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:16.757317 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] The token cache is being invalidate because project d01bf2944a9e43398b511ab79908e211 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:49:16.757871 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:16.758896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:16.767935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 727/1469] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:49:15 2022] DELETE /identity/v3/projects/d01bf2944a9e43398b511ab79908e211 => generated 0 bytes in 833 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:49:16.769049 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:16.770588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:16.880006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:16.921859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71875ec8-08b1-4e85-b4c2-1e52a50774ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:16.923071 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71875ec8-08b1-4e85-b4c2-1e52a50774ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:16.923545 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71875ec8-08b1-4e85-b4c2-1e52a50774ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:16.925564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71875ec8-08b1-4e85-b4c2-1e52a50774ab admin admin] PATH_INFO: `/v3/projects/a077efef66d24d7aa9f5f71d3d9c7360` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:16.941726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71875ec8-08b1-4e85-b4c2-1e52a50774ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a077efef66d24d7aa9f5f71d3d9c7360)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:16.946421 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71875ec8-08b1-4e85-b4c2-1e52a50774ab admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:17.046049 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9aefa-d349-4bf6-9052-0198ef50e9af admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:17.061257 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febeb17-915c-4fa0-9fd0-f327939162a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:17.061979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febeb17-915c-4fa0-9fd0-f327939162a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:17.062258 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febeb17-915c-4fa0-9fd0-f327939162a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:17.062525 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febeb17-915c-4fa0-9fd0-f327939162a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:17.063213 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febeb17-915c-4fa0-9fd0-f327939162a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:17.065166 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febeb17-915c-4fa0-9fd0-f327939162a1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:17.094372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 728/1470] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:17.095309 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febeb17-915c-4fa0-9fd0-f327939162a1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:17.102898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0febeb17-915c-4fa0-9fd0-f327939162a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:17.103912 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-71875ec8-08b1-4e85-b4c2-1e52a50774ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a077efef66d24d7aa9f5f71d3d9c7360', 'request_id': 'req-71875ec8-08b1-4e85-b4c2-1e52a50774ab'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:17.117639 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-71875ec8-08b1-4e85-b4c2-1e52a50774ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a077efef66d24d7aa9f5f71d3d9c7360', 'request_id': 'req-71875ec8-08b1-4e85-b4c2-1e52a50774ab'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:17.129282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-71875ec8-08b1-4e85-b4c2-1e52a50774ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a077efef66d24d7aa9f5f71d3d9c7360', 'request_id': 'req-71875ec8-08b1-4e85-b4c2-1e52a50774ab'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:17.137329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febeb17-915c-4fa0-9fd0-f327939162a1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:17.145612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:17.161986 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-71875ec8-08b1-4e85-b4c2-1e52a50774ab admin admin] The token cache is being invalidate because project a077efef66d24d7aa9f5f71d3d9c7360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:49:17.164403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-71875ec8-08b1-4e85-b4c2-1e52a50774ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:17.168489 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-71875ec8-08b1-4e85-b4c2-1e52a50774ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:17.170679 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 743/1471] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:49:16 2022] DELETE /identity/v3/projects/a077efef66d24d7aa9f5f71d3d9c7360 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:17.173251 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31a521-17ac-48ca-8f02-f8145df70f39 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:17.179361 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31a521-17ac-48ca-8f02-f8145df70f39 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:17.180010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31a521-17ac-48ca-8f02-f8145df70f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:17.300547 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-2e31a521-17ac-48ca-8f02-f8145df70f39 None None] MFA Rules not processed for user `0b01e277944d4edcb8625ebf4544e340`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:17.303179 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2e31a521-17ac-48ca-8f02-f8145df70f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:17.424556 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3f7b0-ed4c-433d-bc43-8393c52548b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:17.425682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3f7b0-ed4c-433d-bc43-8393c52548b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:17.426171 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3f7b0-ed4c-433d-bc43-8393c52548b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:17.426654 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3f7b0-ed4c-433d-bc43-8393c52548b1 admin admin] PATH_INFO: `/v3/users/854d3e695b354a17989607f3153d8c6b` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:17.433415 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3f7b0-ed4c-433d-bc43-8393c52548b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=854d3e695b354a17989607f3153d8c6b)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:17.437356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3f7b0-ed4c-433d-bc43-8393c52548b1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:17.665024 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 744/1472] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:17.669275 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31a521-17ac-48ca-8f02-f8145df70f39 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:17.673559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2e31a521-17ac-48ca-8f02-f8145df70f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:17.766627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31a521-17ac-48ca-8f02-f8145df70f39 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:17.769947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a1c3f7b0-ed4c-433d-bc43-8393c52548b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '854d3e695b354a17989607f3153d8c6b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:17.776133 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a1c3f7b0-ed4c-433d-bc43-8393c52548b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '854d3e695b354a17989607f3153d8c6b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:17.778660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a1c3f7b0-ed4c-433d-bc43-8393c52548b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '854d3e695b354a17989607f3153d8c6b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:17.784332 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93313f06-aa55-4469-b460-603d75989034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:17.785922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93313f06-aa55-4469-b460-603d75989034 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:17.788921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93313f06-aa55-4469-b460-603d75989034 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:17.790376 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93313f06-aa55-4469-b460-603d75989034 admin admin] PATH_INFO: `/v3/projects/147d477375ed4990ace973b34fcf3811/users/0b78da76f5e94c3b90703c478594d03c/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:17.862333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a1c3f7b0-ed4c-433d-bc43-8393c52548b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '854d3e695b354a17989607f3153d8c6b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:17.873298 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93313f06-aa55-4469-b460-603d75989034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=147d477375ed4990ace973b34fcf3811, user_id=0b78da76f5e94c3b90703c478594d03c, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:17.881949 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93313f06-aa55-4469-b460-603d75989034 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:17.902715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a1c3f7b0-ed4c-433d-bc43-8393c52548b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '854d3e695b354a17989607f3153d8c6b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:17.943859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 729/1473] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:49:17 2022] DELETE /identity/v3/users/854d3e695b354a17989607f3153d8c6b => generated 0 bytes in 849 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:17.952983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8e700-1f79-4cb8-a8c4-3276acd131a6 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:17.953256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8e700-1f79-4cb8-a8c4-3276acd131a6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:17.953659 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8e700-1f79-4cb8-a8c4-3276acd131a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:18.029932 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 745/1474] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:17 2022] PUT /identity/v3/projects/147d477375ed4990ace973b34fcf3811/users/0b78da76f5e94c3b90703c478594d03c/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:18.032230 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-92a8e700-1f79-4cb8-a8c4-3276acd131a6 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:18.033508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-92a8e700-1f79-4cb8-a8c4-3276acd131a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:18.040677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898908d-b22b-4d8a-ba34-3d2ad8dd1571 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:18.054028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898908d-b22b-4d8a-ba34-3d2ad8dd1571 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:18.054028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898908d-b22b-4d8a-ba34-3d2ad8dd1571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:18.181884 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:18.230585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 730/1475] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:18.231704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8e700-1f79-4cb8-a8c4-3276acd131a6 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:18.243702 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-2898908d-b22b-4d8a-ba34-3d2ad8dd1571 None None] MFA Rules not processed for user `10364e42b429400fb014eb5f6d60e3bb`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:18.245383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-92a8e700-1f79-4cb8-a8c4-3276acd131a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:18.250113 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2898908d-b22b-4d8a-ba34-3d2ad8dd1571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:18.264220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8e700-1f79-4cb8-a8c4-3276acd131a6 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:18.285488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d42a6-7436-4aa5-9d33-8f650a72d205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:18.285488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d42a6-7436-4aa5-9d33-8f650a72d205 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:18.285488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d42a6-7436-4aa5-9d33-8f650a72d205 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:18.285488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d42a6-7436-4aa5-9d33-8f650a72d205 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:18.286284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d42a6-7436-4aa5-9d33-8f650a72d205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:18.286836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d42a6-7436-4aa5-9d33-8f650a72d205 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:18.319147 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 731/1476] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:18.319147 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e783f-17f8-4d65-82d9-26821220a759 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:18.319147 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e783f-17f8-4d65-82d9-26821220a759 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:18.319147 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e783f-17f8-4d65-82d9-26821220a759 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:18.323423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 732/1477] 10.222.0.36 () {54 vars in 921 bytes} [Sat Oct 15 13:49:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:49:18.325726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69291d9-7f8e-44e6-a967-cd023819d3d5 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:18.325726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69291d9-7f8e-44e6-a967-cd023819d3d5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:18.326097 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69291d9-7f8e-44e6-a967-cd023819d3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:18.386498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a69291d9-7f8e-44e6-a967-cd023819d3d5 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:18.393966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a69291d9-7f8e-44e6-a967-cd023819d3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:18.423553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 733/1478] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:18.423553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69291d9-7f8e-44e6-a967-cd023819d3d5 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:18.458995 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69291d9-7f8e-44e6-a967-cd023819d3d5 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:18.478812 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b39aae-e386-4ded-942d-572f735d8b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:18.480894 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b39aae-e386-4ded-942d-572f735d8b53 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:18.481401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b39aae-e386-4ded-942d-572f735d8b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:18.484344 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b39aae-e386-4ded-942d-572f735d8b53 admin admin] PATH_INFO: `/v3/projects/147d477375ed4990ace973b34fcf3811/users/0b78da76f5e94c3b90703c478594d03c/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:18.492582 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b39aae-e386-4ded-942d-572f735d8b53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=147d477375ed4990ace973b34fcf3811, user_id=0b78da76f5e94c3b90703c478594d03c, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:18.501544 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b39aae-e386-4ded-942d-572f735d8b53 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:18.548151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 734/1479] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:18 2022] PUT /identity/v3/projects/147d477375ed4990ace973b34fcf3811/users/0b78da76f5e94c3b90703c478594d03c/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:18.548151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b39aae-e386-4ded-942d-572f735d8b53 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:18.656166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 746/1480] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 624 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:18.674533 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b18e67-da42-4f77-9653-06433c39694c None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:18.675159 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b18e67-da42-4f77-9653-06433c39694c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:18.675650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b18e67-da42-4f77-9653-06433c39694c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:18.807260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-b8b18e67-da42-4f77-9653-06433c39694c None None] MFA Rules not processed for user `2e13692c1e83413a909d99bb7db2e4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:18.829891 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b8b18e67-da42-4f77-9653-06433c39694c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:18.923469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b39aae-e386-4ded-942d-572f735d8b53 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:18.946557 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74533dc5-f3e1-489f-9a39-19da09126217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:18.948307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74533dc5-f3e1-489f-9a39-19da09126217 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:18.949249 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74533dc5-f3e1-489f-9a39-19da09126217 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:18.950020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74533dc5-f3e1-489f-9a39-19da09126217 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:18.951949 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74533dc5-f3e1-489f-9a39-19da09126217 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:49:18.954916 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74533dc5-f3e1-489f-9a39-19da09126217 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:18.962315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74533dc5-f3e1-489f-9a39-19da09126217 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:19.015578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 735/1481] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:49:19.021951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f70d9-e25d-4760-8819-75158115503d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:19.023013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f70d9-e25d-4760-8819-75158115503d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:19.024499 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f70d9-e25d-4760-8819-75158115503d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:19.109041 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:19.118184 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-459f70d9-e25d-4760-8819-75158115503d None None] MFA Rules not processed for user `0b78da76f5e94c3b90703c478594d03c`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:19.123029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-459f70d9-e25d-4760-8819-75158115503d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:19.161298 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 747/1482] 10.222.0.36 () {58 vars in 1018 bytes} [Sat Oct 15 13:49:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 504 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:19.163951 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b18e67-da42-4f77-9653-06433c39694c None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:19.168416 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b8b18e67-da42-4f77-9653-06433c39694c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:19.219226 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b18e67-da42-4f77-9653-06433c39694c None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:19.230253 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b269af-e309-43e1-b468-42d29e79815d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:19.232065 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b269af-e309-43e1-b468-42d29e79815d admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:19.233221 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b269af-e309-43e1-b468-42d29e79815d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:19.234045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b269af-e309-43e1-b468-42d29e79815d admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:19.235388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b269af-e309-43e1-b468-42d29e79815d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:19.240271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b269af-e309-43e1-b468-42d29e79815d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:19.303955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 748/1483] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:19 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:19.306434 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b269af-e309-43e1-b468-42d29e79815d admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:19.355023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 736/1484] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:19.359246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f70d9-e25d-4760-8819-75158115503d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:19.405749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f70d9-e25d-4760-8819-75158115503d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:19.506557 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b269af-e309-43e1-b468-42d29e79815d admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:19.560238 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:19.570801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03bf005-af68-4889-a13c-adfdfc118909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:19.578434 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bf005-af68-4889-a13c-adfdfc118909 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:19.580536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bf005-af68-4889-a13c-adfdfc118909 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:19.583498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bf005-af68-4889-a13c-adfdfc118909 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:19.587273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03bf005-af68-4889-a13c-adfdfc118909 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:19.605466 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03bf005-af68-4889-a13c-adfdfc118909 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:19.694488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb6f29-3349-48dd-85b6-2b32e8111074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:19.697231 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb6f29-3349-48dd-85b6-2b32e8111074 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:19.697704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb6f29-3349-48dd-85b6-2b32e8111074 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:19.698235 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb6f29-3349-48dd-85b6-2b32e8111074 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:19.699612 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb6f29-3349-48dd-85b6-2b32e8111074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:19.703485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb6f29-3349-48dd-85b6-2b32e8111074 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:19.751035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 749/1485] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:19 2022] POST /identity/v3/projects => generated 369 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:19.752572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03bf005-af68-4889-a13c-adfdfc118909 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:19.760553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-d03bf005-af68-4889-a13c-adfdfc118909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:19.796075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 737/1486] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:19 2022] POST /identity/v3/users => generated 293 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:19.801681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb6f29-3349-48dd-85b6-2b32e8111074 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:19.843907 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb6f29-3349-48dd-85b6-2b32e8111074 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:19.981809 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03bf005-af68-4889-a13c-adfdfc118909 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:19.994105 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965ca50-68a9-4926-a840-c4cc16ec248d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:19.994977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965ca50-68a9-4926-a840-c4cc16ec248d service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:19.996242 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965ca50-68a9-4926-a840-c4cc16ec248d service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:19.996887 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965ca50-68a9-4926-a840-c4cc16ec248d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:20.142545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf15a97-30bb-496d-b0e5-1c6bbcada959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:20.143560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf15a97-30bb-496d-b0e5-1c6bbcada959 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:20.144396 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf15a97-30bb-496d-b0e5-1c6bbcada959 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:20.144858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf15a97-30bb-496d-b0e5-1c6bbcada959 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:20.150606 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf15a97-30bb-496d-b0e5-1c6bbcada959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:20.161196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf15a97-30bb-496d-b0e5-1c6bbcada959 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:20.210236 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965ca50-68a9-4926-a840-c4cc16ec248d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:20.214185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965ca50-68a9-4926-a840-c4cc16ec248d service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:20.245717 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 738/1487] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:19 2022] POST /identity/v3/users => generated 285 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:20.248493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf15a97-30bb-496d-b0e5-1c6bbcada959 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:20.292312 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 750/1488] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:20.294128 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965ca50-68a9-4926-a840-c4cc16ec248d service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:20.296035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf15a97-30bb-496d-b0e5-1c6bbcada959 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:20.337360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e26b0f-ef64-485e-af32-2bd4de48ccac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:20.350309 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e26b0f-ef64-485e-af32-2bd4de48ccac admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:20.351392 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e26b0f-ef64-485e-af32-2bd4de48ccac admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:20.351392 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e26b0f-ef64-485e-af32-2bd4de48ccac admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:20.352547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e26b0f-ef64-485e-af32-2bd4de48ccac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:20.359849 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e26b0f-ef64-485e-af32-2bd4de48ccac admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:20.370075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965ca50-68a9-4926-a840-c4cc16ec248d service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:20.394171 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cbc6fa-9960-461e-befe-c050695ca12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:20.395262 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbc6fa-9960-461e-befe-c050695ca12b admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:20.396082 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbc6fa-9960-461e-befe-c050695ca12b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:20.396262 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbc6fa-9960-461e-befe-c050695ca12b admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:20.397397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cbc6fa-9960-461e-befe-c050695ca12b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:20.412984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cbc6fa-9960-461e-befe-c050695ca12b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:20.427613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 739/1489] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:20.430343 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e26b0f-ef64-485e-af32-2bd4de48ccac admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:20.452556 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e26b0f-ef64-485e-af32-2bd4de48ccac admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:20.464128 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e351e-19a8-49ed-aa92-d48eabecc1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:20.465246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e351e-19a8-49ed-aa92-d48eabecc1af admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:20.465715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e351e-19a8-49ed-aa92-d48eabecc1af admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:20.466244 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e351e-19a8-49ed-aa92-d48eabecc1af admin admin] PATH_INFO: `/v3/projects/a6ebf4f6aff24a52b2a576688f0ed547/users/b9335df2fedc4f09a654fe8ed5b1c3e9/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:20.492240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 751/1490] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:20.494644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cbc6fa-9960-461e-befe-c050695ca12b admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:20.517498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e351e-19a8-49ed-aa92-d48eabecc1af admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ebf4f6aff24a52b2a576688f0ed547, user_id=b9335df2fedc4f09a654fe8ed5b1c3e9, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:20.526172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:20.527939 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e351e-19a8-49ed-aa92-d48eabecc1af admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:20.570639 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cbc6fa-9960-461e-befe-c050695ca12b admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:20.607420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 740/1491] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:20 2022] PUT /identity/v3/projects/a6ebf4f6aff24a52b2a576688f0ed547/users/b9335df2fedc4f09a654fe8ed5b1c3e9/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:20.619962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e351e-19a8-49ed-aa92-d48eabecc1af admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:20.942607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399804f-4103-45b5-964a-f4c89cccf47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:20.944089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399804f-4103-45b5-964a-f4c89cccf47e admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:20.944982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399804f-4103-45b5-964a-f4c89cccf47e admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:20.945427 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399804f-4103-45b5-964a-f4c89cccf47e admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:20.946295 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399804f-4103-45b5-964a-f4c89cccf47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:20.952932 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399804f-4103-45b5-964a-f4c89cccf47e admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:20.983653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e351e-19a8-49ed-aa92-d48eabecc1af admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:20.985798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 752/1492] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:20.989441 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399804f-4103-45b5-964a-f4c89cccf47e admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:20.996568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379659f-1eec-4a21-96d9-ba1463f8e9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:20.997949 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379659f-1eec-4a21-96d9-ba1463f8e9eb admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:20.998574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379659f-1eec-4a21-96d9-ba1463f8e9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:20.999114 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379659f-1eec-4a21-96d9-ba1463f8e9eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:20.999965 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379659f-1eec-4a21-96d9-ba1463f8e9eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:21.006939 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379659f-1eec-4a21-96d9-ba1463f8e9eb admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:21.014841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399804f-4103-45b5-964a-f4c89cccf47e admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:21.030957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 741/1493] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:21.037943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379659f-1eec-4a21-96d9-ba1463f8e9eb admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:21.047562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418a5b0-7438-4854-9679-126cd3a64bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:21.050198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418a5b0-7438-4854-9679-126cd3a64bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:21.051418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418a5b0-7438-4854-9679-126cd3a64bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:21.051725 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418a5b0-7438-4854-9679-126cd3a64bdf admin admin] PATH_INFO: `/v3/projects/8dd8bbe63f614ffcbfcfcaa67ce42d12/users/cbb04b66ff4c4e829855788428e01007/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:21.128167 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379659f-1eec-4a21-96d9-ba1463f8e9eb admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:21.158961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44b750-97e1-4b09-ba82-99186500d73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:21.165070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44b750-97e1-4b09-ba82-99186500d73b admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:21.165930 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44b750-97e1-4b09-ba82-99186500d73b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:21.166138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44b750-97e1-4b09-ba82-99186500d73b admin admin] PATH_INFO: `/v3/projects/a6ebf4f6aff24a52b2a576688f0ed547/users/b9335df2fedc4f09a654fe8ed5b1c3e9/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:21.172878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418a5b0-7438-4854-9679-126cd3a64bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd8bbe63f614ffcbfcfcaa67ce42d12, user_id=cbb04b66ff4c4e829855788428e01007, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:21.176991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d44b750-97e1-4b09-ba82-99186500d73b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ebf4f6aff24a52b2a576688f0ed547, user_id=b9335df2fedc4f09a654fe8ed5b1c3e9, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:21.181152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418a5b0-7438-4854-9679-126cd3a64bdf admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:21.188121 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d44b750-97e1-4b09-ba82-99186500d73b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:21.237523 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 753/1494] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:20 2022] PUT /identity/v3/projects/8dd8bbe63f614ffcbfcfcaa67ce42d12/users/cbb04b66ff4c4e829855788428e01007/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:21.242631 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 742/1495] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:21 2022] PUT /identity/v3/projects/a6ebf4f6aff24a52b2a576688f0ed547/users/b9335df2fedc4f09a654fe8ed5b1c3e9/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:21.253683 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e486f0-5dca-4488-9e89-1f722f0f43d4 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:21.254005 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e486f0-5dca-4488-9e89-1f722f0f43d4 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:21.254348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e486f0-5dca-4488-9e89-1f722f0f43d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:21.342394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-b4e486f0-5dca-4488-9e89-1f722f0f43d4 None None] MFA Rules not processed for user `cbb04b66ff4c4e829855788428e01007`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:21.347020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b4e486f0-5dca-4488-9e89-1f722f0f43d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:21.635941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 754/1496] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:21.638658 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44b750-97e1-4b09-ba82-99186500d73b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:21.643951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:21.774075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e486f0-5dca-4488-9e89-1f722f0f43d4 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:21.776092 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b4e486f0-5dca-4488-9e89-1f722f0f43d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:21.862383 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44b750-97e1-4b09-ba82-99186500d73b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:21.870807 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b33d77-b8f0-47b6-ae98-51f4a67455a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:21.872903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b33d77-b8f0-47b6-ae98-51f4a67455a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:21.873508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b33d77-b8f0-47b6-ae98-51f4a67455a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:21.874092 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b33d77-b8f0-47b6-ae98-51f4a67455a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:21.875450 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b33d77-b8f0-47b6-ae98-51f4a67455a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:21.892092 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b33d77-b8f0-47b6-ae98-51f4a67455a0 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:21.979599 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 743/1497] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:21 2022] POST /identity/v3/projects => generated 369 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:21.980692 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b33d77-b8f0-47b6-ae98-51f4a67455a0 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:21.991040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-18b33d77-b8f0-47b6-ae98-51f4a67455a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:22.007307 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e486f0-5dca-4488-9e89-1f722f0f43d4 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:22.047820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:22.177498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b33d77-b8f0-47b6-ae98-51f4a67455a0 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:22.180172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:22.180852 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:22.181177 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:22.181398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] PATH_INFO: `/v3/projects/59bbc598003c4f5286374b105c2e592a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:22.182775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59bbc598003c4f5286374b105c2e592a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:22.184096 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0d168-6f1a-41de-a150-0c0cd4e00b54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:22.184851 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:22.185028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0d168-6f1a-41de-a150-0c0cd4e00b54 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:22.185413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0d168-6f1a-41de-a150-0c0cd4e00b54 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:22.185931 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0d168-6f1a-41de-a150-0c0cd4e00b54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:22.197214 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0d168-6f1a-41de-a150-0c0cd4e00b54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:22.199057 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0d168-6f1a-41de-a150-0c0cd4e00b54 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:22.213554 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 744/1498] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:22.214719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0d168-6f1a-41de-a150-0c0cd4e00b54 service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:22.233621 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0d168-6f1a-41de-a150-0c0cd4e00b54 service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:22.254056 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61aefa2-4a3d-49d4-b2ac-61ea60b6b5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:22.255749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aefa2-4a3d-49d4-b2ac-61ea60b6b5dc admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:22.257129 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aefa2-4a3d-49d4-b2ac-61ea60b6b5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:22.257660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aefa2-4a3d-49d4-b2ac-61ea60b6b5dc admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:22.259567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61aefa2-4a3d-49d4-b2ac-61ea60b6b5dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:22.264481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61aefa2-4a3d-49d4-b2ac-61ea60b6b5dc admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:22.282806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59bbc598003c4f5286374b105c2e592a', 'request_id': 'req-474ab9e3-3794-45a6-a39e-29419123f911'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:22.299134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59bbc598003c4f5286374b105c2e592a', 'request_id': 'req-474ab9e3-3794-45a6-a39e-29419123f911'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:22.314124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59bbc598003c4f5286374b105c2e592a', 'request_id': 'req-474ab9e3-3794-45a6-a39e-29419123f911'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:22.317225 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 745/1499] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:22 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:22.318564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61aefa2-4a3d-49d4-b2ac-61ea60b6b5dc admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:22.334973 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] The token cache is being invalidate because project 59bbc598003c4f5286374b105c2e592a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:49:22.335287 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:22.335441 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61aefa2-4a3d-49d4-b2ac-61ea60b6b5dc admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:22.336139 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:22.338071 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 755/1500] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:49:21 2022] DELETE /identity/v3/projects/59bbc598003c4f5286374b105c2e592a => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:22.355979 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:22.360828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:22.464584 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:22.485937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73ef59-f0dc-4474-b048-6fa872cfbff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:22.487372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ef59-f0dc-4474-b048-6fa872cfbff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:22.487687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ef59-f0dc-4474-b048-6fa872cfbff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:22.488188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ef59-f0dc-4474-b048-6fa872cfbff5 admin admin] PATH_INFO: `/v3/users/8467bbcb8ede41b69c67e55b106817c9` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:22.489557 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73ef59-f0dc-4474-b048-6fa872cfbff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8467bbcb8ede41b69c67e55b106817c9)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:22.491747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73ef59-f0dc-4474-b048-6fa872cfbff5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:22.512301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ab9e3-3794-45a6-a39e-29419123f911 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:22.521582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905f97c-794d-4552-93b5-60a268193e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:22.524637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905f97c-794d-4552-93b5-60a268193e9d admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:22.524637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905f97c-794d-4552-93b5-60a268193e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:22.524637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905f97c-794d-4552-93b5-60a268193e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:22.526377 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905f97c-794d-4552-93b5-60a268193e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:22.530235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905f97c-794d-4552-93b5-60a268193e9d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:22.548519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 756/1501] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:22.549878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654e269-c231-47ef-b397-9b7f55c2f988 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:22.550195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654e269-c231-47ef-b397-9b7f55c2f988 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:22.550441 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654e269-c231-47ef-b397-9b7f55c2f988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:22.574555 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-6654e269-c231-47ef-b397-9b7f55c2f988 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:22.576423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6654e269-c231-47ef-b397-9b7f55c2f988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:22.585953 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 757/1502] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:22.586847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654e269-c231-47ef-b397-9b7f55c2f988 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:22.590197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1b73ef59-f0dc-4474-b048-6fa872cfbff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8467bbcb8ede41b69c67e55b106817c9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:22.591443 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1b73ef59-f0dc-4474-b048-6fa872cfbff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8467bbcb8ede41b69c67e55b106817c9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:22.593300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1b73ef59-f0dc-4474-b048-6fa872cfbff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8467bbcb8ede41b69c67e55b106817c9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:22.598741 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654e269-c231-47ef-b397-9b7f55c2f988 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:22.613243 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0cd32-c782-4a89-8bc3-c241a2eec612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:22.616621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0cd32-c782-4a89-8bc3-c241a2eec612 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:22.617112 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0cd32-c782-4a89-8bc3-c241a2eec612 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:22.617768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0cd32-c782-4a89-8bc3-c241a2eec612 admin admin] PATH_INFO: `/v3/projects/3904810af867403b91a0e51f5e5c5e40/users/ed509a2359254f29bbfd5d09cc9ab06d/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:22.641963 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1b73ef59-f0dc-4474-b048-6fa872cfbff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8467bbcb8ede41b69c67e55b106817c9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:22.665377 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0cd32-c782-4a89-8bc3-c241a2eec612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3904810af867403b91a0e51f5e5c5e40, user_id=ed509a2359254f29bbfd5d09cc9ab06d, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:22.670206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1b73ef59-f0dc-4474-b048-6fa872cfbff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8467bbcb8ede41b69c67e55b106817c9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:22.673632 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0cd32-c782-4a89-8bc3-c241a2eec612 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:22.715652 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 758/1503] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:22 2022] PUT /identity/v3/projects/3904810af867403b91a0e51f5e5c5e40/users/ed509a2359254f29bbfd5d09cc9ab06d/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:22.720878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe011a4-08bb-4ef5-ad83-7703def23087 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:22.721535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe011a4-08bb-4ef5-ad83-7703def23087 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:22.722012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe011a4-08bb-4ef5-ad83-7703def23087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:22.723808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 746/1504] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:49:22 2022] DELETE /identity/v3/users/8467bbcb8ede41b69c67e55b106817c9 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:22.729852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73ef59-f0dc-4474-b048-6fa872cfbff5 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:22.734287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1b73ef59-f0dc-4474-b048-6fa872cfbff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:22.769962 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:22.782612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-5fe011a4-08bb-4ef5-ad83-7703def23087 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:22.788235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-5fe011a4-08bb-4ef5-ad83-7703def23087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:22.958871 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73ef59-f0dc-4474-b048-6fa872cfbff5 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:22.969928 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b820e8b-398f-4c25-b29f-5a3724ad592a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:22.970912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b820e8b-398f-4c25-b29f-5a3724ad592a admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:22.971351 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b820e8b-398f-4c25-b29f-5a3724ad592a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:22.972065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b820e8b-398f-4c25-b29f-5a3724ad592a admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:22.973163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b820e8b-398f-4c25-b29f-5a3724ad592a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:22.976027 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b820e8b-398f-4c25-b29f-5a3724ad592a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:22.999233 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 747/1505] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:23.001548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc4fd4-be15-43f6-983f-e4929ecb1948 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:23.002067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc4fd4-be15-43f6-983f-e4929ecb1948 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:23.002747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc4fd4-be15-43f6-983f-e4929ecb1948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:23.006853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 759/1506] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:23.006853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe011a4-08bb-4ef5-ad83-7703def23087 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:23.032263 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-59fc4fd4-be15-43f6-983f-e4929ecb1948 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:23.033967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-59fc4fd4-be15-43f6-983f-e4929ecb1948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:23.041005 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe011a4-08bb-4ef5-ad83-7703def23087 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:23.044473 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 748/1507] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:23.045719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc4fd4-be15-43f6-983f-e4929ecb1948 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:23.056932 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea0766-a84a-4095-bf22-e9e4fbec1c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:23.058178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea0766-a84a-4095-bf22-e9e4fbec1c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:23.058623 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea0766-a84a-4095-bf22-e9e4fbec1c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:23.059077 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea0766-a84a-4095-bf22-e9e4fbec1c2b admin admin] PATH_INFO: `/v3/projects/3904810af867403b91a0e51f5e5c5e40/users/ed509a2359254f29bbfd5d09cc9ab06d/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:23.063553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea0766-a84a-4095-bf22-e9e4fbec1c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3904810af867403b91a0e51f5e5c5e40, user_id=ed509a2359254f29bbfd5d09cc9ab06d, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:23.068841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea0766-a84a-4095-bf22-e9e4fbec1c2b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:23.073876 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc4fd4-be15-43f6-983f-e4929ecb1948 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:23.082238 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d8d7e-d489-4279-84cc-8725df0e580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:23.083392 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8d7e-d489-4279-84cc-8725df0e580e admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:23.083854 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8d7e-d489-4279-84cc-8725df0e580e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:23.084312 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8d7e-d489-4279-84cc-8725df0e580e admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:23.085597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d8d7e-d489-4279-84cc-8725df0e580e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:49:23.086085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d8d7e-d489-4279-84cc-8725df0e580e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:23.088241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 760/1508] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:23 2022] PUT /identity/v3/projects/3904810af867403b91a0e51f5e5c5e40/users/ed509a2359254f29bbfd5d09cc9ab06d/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:23.089021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d8d7e-d489-4279-84cc-8725df0e580e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:23.089620 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea0766-a84a-4095-bf22-e9e4fbec1c2b admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:23.093585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-22ea0766-a84a-4095-bf22-e9e4fbec1c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:23.123908 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 749/1509] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:49:23.126425 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118605d4-7d42-415f-9f91-ee0cbf73bb38 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:23.127066 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118605d4-7d42-415f-9f91-ee0cbf73bb38 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:23.127636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118605d4-7d42-415f-9f91-ee0cbf73bb38 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:23.130819 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 750/1510] 10.222.0.36 () {54 vars in 921 bytes} [Sat Oct 15 13:49:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:49:23.133296 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77ba04-18b2-456b-a984-16d5e9f50d30 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:23.134211 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77ba04-18b2-456b-a984-16d5e9f50d30 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:23.134739 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77ba04-18b2-456b-a984-16d5e9f50d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:23.211360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5c77ba04-18b2-456b-a984-16d5e9f50d30 None None] MFA Rules not processed for user `ed509a2359254f29bbfd5d09cc9ab06d`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:23.212697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5c77ba04-18b2-456b-a984-16d5e9f50d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:23.222059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea0766-a84a-4095-bf22-e9e4fbec1c2b admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:23.232480 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e1924-9908-49b1-990b-ff864d261140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:23.233766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e1924-9908-49b1-990b-ff864d261140 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:23.234111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e1924-9908-49b1-990b-ff864d261140 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:23.234390 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e1924-9908-49b1-990b-ff864d261140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:23.326219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:23.404979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 751/1511] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:23.404979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77ba04-18b2-456b-a984-16d5e9f50d30 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:23.416814 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e1924-9908-49b1-990b-ff864d261140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:23.418536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e1924-9908-49b1-990b-ff864d261140 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:23.437987 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77ba04-18b2-456b-a984-16d5e9f50d30 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:23.442997 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 761/1512] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:23.445602 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80416b46-23ce-434f-b06c-7493693346bf None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:23.445882 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80416b46-23ce-434f-b06c-7493693346bf None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:23.446152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80416b46-23ce-434f-b06c-7493693346bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:23.449228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f982e36-14ee-4422-a20e-9ef7b4ea8197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:23.451751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f982e36-14ee-4422-a20e-9ef7b4ea8197 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:23.452580 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f982e36-14ee-4422-a20e-9ef7b4ea8197 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:23.453247 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f982e36-14ee-4422-a20e-9ef7b4ea8197 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:23.455784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f982e36-14ee-4422-a20e-9ef7b4ea8197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:23.464495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f982e36-14ee-4422-a20e-9ef7b4ea8197 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:23.489481 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-80416b46-23ce-434f-b06c-7493693346bf None None] MFA Rules not processed for user `2e13692c1e83413a909d99bb7db2e4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:23.490845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-80416b46-23ce-434f-b06c-7493693346bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:23.548784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 752/1513] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:23 2022] POST /identity/v3/projects => generated 357 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:23.549856 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f982e36-14ee-4422-a20e-9ef7b4ea8197 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:23.554535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9f982e36-14ee-4422-a20e-9ef7b4ea8197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:23.650424 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:23.714622 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 762/1514] 10.222.0.36 () {58 vars in 1018 bytes} [Sat Oct 15 13:49:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:23.716389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80416b46-23ce-434f-b06c-7493693346bf None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:23.730444 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f982e36-14ee-4422-a20e-9ef7b4ea8197 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:23.746254 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9a206-6d8a-4128-93d9-24222b4405a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:23.748060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9a206-6d8a-4128-93d9-24222b4405a1 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:23.748990 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9a206-6d8a-4128-93d9-24222b4405a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:23.749969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9a206-6d8a-4128-93d9-24222b4405a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:23.754416 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80416b46-23ce-434f-b06c-7493693346bf None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:23.977454 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9a206-6d8a-4128-93d9-24222b4405a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:23.980681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9a206-6d8a-4128-93d9-24222b4405a1 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:24.018384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d042e-9341-45e6-8e4a-aa98327e6020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:24.019538 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d042e-9341-45e6-8e4a-aa98327e6020 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:24.020058 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d042e-9341-45e6-8e4a-aa98327e6020 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:24.020521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d042e-9341-45e6-8e4a-aa98327e6020 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:24.021684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d042e-9341-45e6-8e4a-aa98327e6020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:24.023372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 753/1515] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:24.036948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d042e-9341-45e6-8e4a-aa98327e6020 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:24.066620 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9a206-6d8a-4128-93d9-24222b4405a1 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:24.069516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cdf9a206-6d8a-4128-93d9-24222b4405a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:24.127090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 763/1516] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:23 2022] POST /identity/v3/users => generated 279 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:24.132627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d042e-9341-45e6-8e4a-aa98327e6020 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:24.155232 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d042e-9341-45e6-8e4a-aa98327e6020 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:24.163281 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453f0bc-0469-46f4-b058-9fa0de2ea998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:24.164116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453f0bc-0469-46f4-b058-9fa0de2ea998 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:24.164579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453f0bc-0469-46f4-b058-9fa0de2ea998 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:24.164949 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453f0bc-0469-46f4-b058-9fa0de2ea998 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:24.165934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453f0bc-0469-46f4-b058-9fa0de2ea998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:24.168966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453f0bc-0469-46f4-b058-9fa0de2ea998 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:24.188329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 764/1517] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:24.192310 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453f0bc-0469-46f4-b058-9fa0de2ea998 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:24.208831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453f0bc-0469-46f4-b058-9fa0de2ea998 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:24.219766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a2f7f-9033-4581-b96c-51d9d21d4cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:24.221447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a2f7f-9033-4581-b96c-51d9d21d4cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:24.221842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a2f7f-9033-4581-b96c-51d9d21d4cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:24.222209 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a2f7f-9033-4581-b96c-51d9d21d4cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:24.223083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a2f7f-9033-4581-b96c-51d9d21d4cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:24.225975 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a2f7f-9033-4581-b96c-51d9d21d4cb0 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:24.245468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 765/1518] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:49:24.251045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a2f7f-9033-4581-b96c-51d9d21d4cb0 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:24.284313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a2f7f-9033-4581-b96c-51d9d21d4cb0 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:24.295575 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3453b-ad04-4f8c-b1b4-330f7f789a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:24.296977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3453b-ad04-4f8c-b1b4-330f7f789a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:24.297256 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3453b-ad04-4f8c-b1b4-330f7f789a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:24.297519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3453b-ad04-4f8c-b1b4-330f7f789a20 admin admin] PATH_INFO: `/v3/projects/3587b1ffcb3f4067804b1eb4d873daf3/users/0692c115a2264edda61b9373bf1b7cf6/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:24.324465 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:24.336843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9a206-6d8a-4128-93d9-24222b4405a1 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:24.352992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569ca41-d1f8-47b7-aaaa-f20d5b8efa41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:24.353488 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3453b-ad04-4f8c-b1b4-330f7f789a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3587b1ffcb3f4067804b1eb4d873daf3, user_id=0692c115a2264edda61b9373bf1b7cf6, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:24.355422 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569ca41-d1f8-47b7-aaaa-f20d5b8efa41 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:24.357255 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569ca41-d1f8-47b7-aaaa-f20d5b8efa41 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:24.358328 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3453b-ad04-4f8c-b1b4-330f7f789a20 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:24.358914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569ca41-d1f8-47b7-aaaa-f20d5b8efa41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:24.373828 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569ca41-d1f8-47b7-aaaa-f20d5b8efa41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:24.375992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569ca41-d1f8-47b7-aaaa-f20d5b8efa41 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:24.403384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 766/1519] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:24 2022] PUT /identity/v3/projects/3587b1ffcb3f4067804b1eb4d873daf3/users/0692c115a2264edda61b9373bf1b7cf6/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:24.409304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13312c56-ff9b-4594-b8db-f8dd6d953094 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:24.410327 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13312c56-ff9b-4594-b8db-f8dd6d953094 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:24.410614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13312c56-ff9b-4594-b8db-f8dd6d953094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:24.414344 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 754/1520] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:24.448041 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-13312c56-ff9b-4594-b8db-f8dd6d953094 None None] MFA Rules not processed for user `0692c115a2264edda61b9373bf1b7cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:24.449430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-13312c56-ff9b-4594-b8db-f8dd6d953094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:24.613623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569ca41-d1f8-47b7-aaaa-f20d5b8efa41 service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:24.625096 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 767/1521] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:24.645850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569ca41-d1f8-47b7-aaaa-f20d5b8efa41 service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:24.746727 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13312c56-ff9b-4594-b8db-f8dd6d953094 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:24.749800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-13312c56-ff9b-4594-b8db-f8dd6d953094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:24.774398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:24.827560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b510f-3def-46cc-a649-bd8f5073df56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:24.828438 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b510f-3def-46cc-a649-bd8f5073df56 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:24.828893 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b510f-3def-46cc-a649-bd8f5073df56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:24.829209 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b510f-3def-46cc-a649-bd8f5073df56 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:24.830066 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b510f-3def-46cc-a649-bd8f5073df56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:24.832649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b510f-3def-46cc-a649-bd8f5073df56 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:24.878439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 755/1522] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:24 2022] POST /identity/v3/projects => generated 355 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:24.878439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acab0a-f198-4806-bbbe-6a83ce307502 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:24.878439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acab0a-f198-4806-bbbe-6a83ce307502 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:24.878439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acab0a-f198-4806-bbbe-6a83ce307502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:24.958137 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f6acab0a-f198-4806-bbbe-6a83ce307502 None None] MFA Rules not processed for user `b9335df2fedc4f09a654fe8ed5b1c3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:24.960792 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f6acab0a-f198-4806-bbbe-6a83ce307502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:24.985956 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13312c56-ff9b-4594-b8db-f8dd6d953094 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:25.162696 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:25.163323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:25.163655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:25.164026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] PATH_INFO: `/v3/projects/a65e5b026e40466fb3d657d4cfa732b5` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:25.172853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a65e5b026e40466fb3d657d4cfa732b5)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:25.175279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:25.235152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 756/1523] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:25.239031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acab0a-f198-4806-bbbe-6a83ce307502 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:25.258225 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acab0a-f198-4806-bbbe-6a83ce307502 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:25.269745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52162740-6591-48fa-80ce-4b22d10e1fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:25.270940 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52162740-6591-48fa-80ce-4b22d10e1fff admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:25.271477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52162740-6591-48fa-80ce-4b22d10e1fff admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:25.272173 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52162740-6591-48fa-80ce-4b22d10e1fff admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:25.273240 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52162740-6591-48fa-80ce-4b22d10e1fff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:25.276087 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52162740-6591-48fa-80ce-4b22d10e1fff admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:25.354330 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a65e5b026e40466fb3d657d4cfa732b5', 'request_id': 'req-2a27de29-2e46-4951-b40a-2cd36b5ddd54'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:25.361723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 757/1524] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:25 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:25.364437 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52162740-6591-48fa-80ce-4b22d10e1fff admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:25.367798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-52162740-6591-48fa-80ce-4b22d10e1fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:25.390008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a65e5b026e40466fb3d657d4cfa732b5', 'request_id': 'req-2a27de29-2e46-4951-b40a-2cd36b5ddd54'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:25.410756 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a65e5b026e40466fb3d657d4cfa732b5', 'request_id': 'req-2a27de29-2e46-4951-b40a-2cd36b5ddd54'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:25.431679 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] The token cache is being invalidate because project a65e5b026e40466fb3d657d4cfa732b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:49:25.434971 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:25.444876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:25.451391 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 768/1525] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:49:24 2022] DELETE /identity/v3/projects/a65e5b026e40466fb3d657d4cfa732b5 => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:25.451391 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:25.454646 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:25.487630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:25.535284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52162740-6591-48fa-80ce-4b22d10e1fff admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:25.592903 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:25.632147 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27de29-2e46-4951-b40a-2cd36b5ddd54 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:25.651429 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ea3db-fcda-46ab-a6c4-14682faea1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:25.652537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ea3db-fcda-46ab-a6c4-14682faea1bc admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:25.652988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ea3db-fcda-46ab-a6c4-14682faea1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:25.653174 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ea3db-fcda-46ab-a6c4-14682faea1bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:25.654116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ea3db-fcda-46ab-a6c4-14682faea1bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:25.658860 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ea3db-fcda-46ab-a6c4-14682faea1bc admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:25.662857 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156cbc4-d054-45d0-b2ba-267d0a525dcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:25.663552 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156cbc4-d054-45d0-b2ba-267d0a525dcc service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:25.663966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156cbc4-d054-45d0-b2ba-267d0a525dcc service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:25.664282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156cbc4-d054-45d0-b2ba-267d0a525dcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:25.683858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a156cbc4-d054-45d0-b2ba-267d0a525dcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:25.708494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 769/1526] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:25.711570 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ea3db-fcda-46ab-a6c4-14682faea1bc admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:25.729048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-456ea3db-fcda-46ab-a6c4-14682faea1bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:25.762861 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ea3db-fcda-46ab-a6c4-14682faea1bc admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:25.789021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c2819-a39d-4eaa-ae4d-6c5dd2f7fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:25.789705 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c2819-a39d-4eaa-ae4d-6c5dd2f7fa62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:25.790011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c2819-a39d-4eaa-ae4d-6c5dd2f7fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:25.790259 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c2819-a39d-4eaa-ae4d-6c5dd2f7fa62 admin admin] PATH_INFO: `/v3/users/0b78da76f5e94c3b90703c478594d03c` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:25.791989 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c2819-a39d-4eaa-ae4d-6c5dd2f7fa62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b78da76f5e94c3b90703c478594d03c)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:25.793779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c2819-a39d-4eaa-ae4d-6c5dd2f7fa62 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:25.834319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156cbc4-d054-45d0-b2ba-267d0a525dcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:25.838177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156cbc4-d054-45d0-b2ba-267d0a525dcc service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:25.877836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 758/1527] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:25.878888 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156cbc4-d054-45d0-b2ba-267d0a525dcc service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:25.912679 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-951c2819-a39d-4eaa-ae4d-6c5dd2f7fa62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b78da76f5e94c3b90703c478594d03c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:25.929389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-951c2819-a39d-4eaa-ae4d-6c5dd2f7fa62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b78da76f5e94c3b90703c478594d03c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:25.931029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-951c2819-a39d-4eaa-ae4d-6c5dd2f7fa62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b78da76f5e94c3b90703c478594d03c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:25.932360 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-951c2819-a39d-4eaa-ae4d-6c5dd2f7fa62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b78da76f5e94c3b90703c478594d03c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:25.933721 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156cbc4-d054-45d0-b2ba-267d0a525dcc service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:25.956026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-951c2819-a39d-4eaa-ae4d-6c5dd2f7fa62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b78da76f5e94c3b90703c478594d03c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:25.956983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05cfaa-4fb9-477e-ac78-39264c30c1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:25.960239 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05cfaa-4fb9-477e-ac78-39264c30c1d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:25.961082 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05cfaa-4fb9-477e-ac78-39264c30c1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:25.961811 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05cfaa-4fb9-477e-ac78-39264c30c1d7 admin admin] PATH_INFO: `/v3/projects/c1dbf9e4805b4a3ca46ce2440778203e/users/d68ae56243644473a9aef337c5a3eca7/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:25.989834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 770/1528] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:49:25 2022] DELETE /identity/v3/users/0b78da76f5e94c3b90703c478594d03c => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:26.003911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05cfaa-4fb9-477e-ac78-39264c30c1d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1dbf9e4805b4a3ca46ce2440778203e, user_id=d68ae56243644473a9aef337c5a3eca7, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:26.011006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05cfaa-4fb9-477e-ac78-39264c30c1d7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:26.054522 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 759/1529] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:25 2022] PUT /identity/v3/projects/c1dbf9e4805b4a3ca46ce2440778203e/users/d68ae56243644473a9aef337c5a3eca7/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:26.056749 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c2819-a39d-4eaa-ae4d-6c5dd2f7fa62 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:26.058404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-951c2819-a39d-4eaa-ae4d-6c5dd2f7fa62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:26.230572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c2819-a39d-4eaa-ae4d-6c5dd2f7fa62 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:26.364081 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550a064-743d-4614-87b5-47bedb62a2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:26.367325 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550a064-743d-4614-87b5-47bedb62a2a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:26.367651 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550a064-743d-4614-87b5-47bedb62a2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:26.369351 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550a064-743d-4614-87b5-47bedb62a2a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:26.369981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550a064-743d-4614-87b5-47bedb62a2a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:26.374727 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550a064-743d-4614-87b5-47bedb62a2a8 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:26.381589 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:26.398794 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 771/1530] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:49:26.404392 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05cfaa-4fb9-477e-ac78-39264c30c1d7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:26.436040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05cfaa-4fb9-477e-ac78-39264c30c1d7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:26.444488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec14de6-d5c9-422b-ba37-4b160fa7a511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:26.445653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec14de6-d5c9-422b-ba37-4b160fa7a511 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:26.446139 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec14de6-d5c9-422b-ba37-4b160fa7a511 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:26.446601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec14de6-d5c9-422b-ba37-4b160fa7a511 admin admin] PATH_INFO: `/v3/projects/c1dbf9e4805b4a3ca46ce2440778203e/users/d68ae56243644473a9aef337c5a3eca7/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:26.450386 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec14de6-d5c9-422b-ba37-4b160fa7a511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1dbf9e4805b4a3ca46ce2440778203e, user_id=d68ae56243644473a9aef337c5a3eca7, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:26.459416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec14de6-d5c9-422b-ba37-4b160fa7a511 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:26.509043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 760/1531] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:26 2022] PUT /identity/v3/projects/c1dbf9e4805b4a3ca46ce2440778203e/users/d68ae56243644473a9aef337c5a3eca7/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:26.512855 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2c48a-1247-49fc-8b3e-a69f50409df0 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:26.513654 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2c48a-1247-49fc-8b3e-a69f50409df0 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:26.514132 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2c48a-1247-49fc-8b3e-a69f50409df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:26.637832 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-20c2c48a-1247-49fc-8b3e-a69f50409df0 None None] MFA Rules not processed for user `d68ae56243644473a9aef337c5a3eca7`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:26.644836 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-20c2c48a-1247-49fc-8b3e-a69f50409df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:26.928528 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 772/1532] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:49:26.970261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec14de6-d5c9-422b-ba37-4b160fa7a511 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:26.974616 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dec14de6-d5c9-422b-ba37-4b160fa7a511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:27.041498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2c48a-1247-49fc-8b3e-a69f50409df0 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:27.043585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-20c2c48a-1247-49fc-8b3e-a69f50409df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:27.144575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec14de6-d5c9-422b-ba37-4b160fa7a511 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:27.152342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbf85a-bcbc-4b84-8b0b-517c6df447f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:27.153563 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbf85a-bcbc-4b84-8b0b-517c6df447f5 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:27.154295 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbf85a-bcbc-4b84-8b0b-517c6df447f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:27.154693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbf85a-bcbc-4b84-8b0b-517c6df447f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:27.173144 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbf85a-bcbc-4b84-8b0b-517c6df447f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:27.175219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbf85a-bcbc-4b84-8b0b-517c6df447f5 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:27.219908 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 761/1533] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:27.243014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2c48a-1247-49fc-8b3e-a69f50409df0 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:27.252736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18775b77-1e90-4816-a737-a861987dfb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:27.253798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18775b77-1e90-4816-a737-a861987dfb4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:27.254456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18775b77-1e90-4816-a737-a861987dfb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:27.255156 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18775b77-1e90-4816-a737-a861987dfb4a admin admin] PATH_INFO: `/v3/projects/147d477375ed4990ace973b34fcf3811` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:27.258469 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18775b77-1e90-4816-a737-a861987dfb4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=147d477375ed4990ace973b34fcf3811)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:27.269201 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18775b77-1e90-4816-a737-a861987dfb4a admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:27.404063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbf85a-bcbc-4b84-8b0b-517c6df447f5 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:27.415315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbf85a-bcbc-4b84-8b0b-517c6df447f5 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:27.434810 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-18775b77-1e90-4816-a737-a861987dfb4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '147d477375ed4990ace973b34fcf3811', 'request_id': 'req-18775b77-1e90-4816-a737-a861987dfb4a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:27.440494 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:27.457543 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-18775b77-1e90-4816-a737-a861987dfb4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '147d477375ed4990ace973b34fcf3811', 'request_id': 'req-18775b77-1e90-4816-a737-a861987dfb4a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:27.469225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-18775b77-1e90-4816-a737-a861987dfb4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '147d477375ed4990ace973b34fcf3811', 'request_id': 'req-18775b77-1e90-4816-a737-a861987dfb4a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:27.492433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-18775b77-1e90-4816-a737-a861987dfb4a admin admin] The token cache is being invalidate because project 147d477375ed4990ace973b34fcf3811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:49:27.492914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-18775b77-1e90-4816-a737-a861987dfb4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:27.494157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-18775b77-1e90-4816-a737-a861987dfb4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:27.496138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 773/1534] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:49:27 2022] DELETE /identity/v3/projects/147d477375ed4990ace973b34fcf3811 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:27.502726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae86f63-9d0c-4d88-8f20-acddd563b6b1 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:27.503176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae86f63-9d0c-4d88-8f20-acddd563b6b1 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:27.503503 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae86f63-9d0c-4d88-8f20-acddd563b6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:27.535408 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-3ae86f63-9d0c-4d88-8f20-acddd563b6b1 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:27.537331 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3ae86f63-9d0c-4d88-8f20-acddd563b6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:27.544061 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35a365-5f27-4f74-bb9d-9cdac4a27abd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:27.545989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35a365-5f27-4f74-bb9d-9cdac4a27abd service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:27.546583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35a365-5f27-4f74-bb9d-9cdac4a27abd service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:27.546862 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35a365-5f27-4f74-bb9d-9cdac4a27abd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:27.549669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fb35a365-5f27-4f74-bb9d-9cdac4a27abd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:27.550252 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:27.626924 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35a365-5f27-4f74-bb9d-9cdac4a27abd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:27.628890 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35a365-5f27-4f74-bb9d-9cdac4a27abd service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:27.643660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 774/1535] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:27.643660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae86f63-9d0c-4d88-8f20-acddd563b6b1 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:27.643660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3ae86f63-9d0c-4d88-8f20-acddd563b6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:27.644144 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 762/1536] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:49:27.645937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684b663-bc7c-42e8-a23a-e481d36542f1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:27.646651 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684b663-bc7c-42e8-a23a-e481d36542f1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:27.647375 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684b663-bc7c-42e8-a23a-e481d36542f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:27.665471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae86f63-9d0c-4d88-8f20-acddd563b6b1 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:27.679647 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b283c-1f00-4bb8-8f77-101b1fe83ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:27.680794 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b283c-1f00-4bb8-8f77-101b1fe83ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:27.680927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-1684b663-bc7c-42e8-a23a-e481d36542f1 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:27.681270 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b283c-1f00-4bb8-8f77-101b1fe83ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:27.681656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b283c-1f00-4bb8-8f77-101b1fe83ae6 admin admin] PATH_INFO: `/v3/users/cbb04b66ff4c4e829855788428e01007` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:27.683208 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1684b663-bc7c-42e8-a23a-e481d36542f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:27.683344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b283c-1f00-4bb8-8f77-101b1fe83ae6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb04b66ff4c4e829855788428e01007)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:27.686206 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b283c-1f00-4bb8-8f77-101b1fe83ae6 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:27.697999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 763/1537] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:49:27.707204 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684b663-bc7c-42e8-a23a-e481d36542f1 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:27.730844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684b663-bc7c-42e8-a23a-e481d36542f1 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:27.740185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b6b89-f72c-4c2b-8c49-3c6fc537cfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:27.741288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b6b89-f72c-4c2b-8c49-3c6fc537cfeb admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:27.741947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b6b89-f72c-4c2b-8c49-3c6fc537cfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:27.742548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b6b89-f72c-4c2b-8c49-3c6fc537cfeb admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:27.743856 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b6b89-f72c-4c2b-8c49-3c6fc537cfeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:49:27.744370 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b6b89-f72c-4c2b-8c49-3c6fc537cfeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:27.748273 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b6b89-f72c-4c2b-8c49-3c6fc537cfeb admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:27.768034 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-314b283c-1f00-4bb8-8f77-101b1fe83ae6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb04b66ff4c4e829855788428e01007'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:27.773414 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 764/1538] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:49:27.778973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b6b89-f72c-4c2b-8c49-3c6fc537cfeb admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:27.791163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-314b283c-1f00-4bb8-8f77-101b1fe83ae6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb04b66ff4c4e829855788428e01007'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:27.792642 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-314b283c-1f00-4bb8-8f77-101b1fe83ae6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb04b66ff4c4e829855788428e01007'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:27.793596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-314b283c-1f00-4bb8-8f77-101b1fe83ae6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb04b66ff4c4e829855788428e01007'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:27.796798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b6b89-f72c-4c2b-8c49-3c6fc537cfeb admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:27.807308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f3bef-2ccb-4c64-902a-d29e42c8b8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:27.808713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f3bef-2ccb-4c64-902a-d29e42c8b8ce admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:27.809117 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f3bef-2ccb-4c64-902a-d29e42c8b8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:27.809525 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f3bef-2ccb-4c64-902a-d29e42c8b8ce admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:27.810247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-314b283c-1f00-4bb8-8f77-101b1fe83ae6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb04b66ff4c4e829855788428e01007'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:27.811772 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f3bef-2ccb-4c64-902a-d29e42c8b8ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:27.817553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f3bef-2ccb-4c64-902a-d29e42c8b8ce admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:27.838171 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 775/1539] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:49:27 2022] DELETE /identity/v3/users/cbb04b66ff4c4e829855788428e01007 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:27.844020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a74ac-6ad8-42a3-8577-89ecc24dcab4 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:27.844665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a74ac-6ad8-42a3-8577-89ecc24dcab4 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:27.845918 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a74ac-6ad8-42a3-8577-89ecc24dcab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:27.883781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 765/1540] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:27 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:27.888033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f3bef-2ccb-4c64-902a-d29e42c8b8ce admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:27.890148 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fa0f3bef-2ccb-4c64-902a-d29e42c8b8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:27.904274 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-465a74ac-6ad8-42a3-8577-89ecc24dcab4 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:27.907299 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-465a74ac-6ad8-42a3-8577-89ecc24dcab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:28.074535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f3bef-2ccb-4c64-902a-d29e42c8b8ce admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:28.080445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96770af6-7a22-479d-bebd-30aba7dee774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:28.081365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96770af6-7a22-479d-bebd-30aba7dee774 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:28.081956 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96770af6-7a22-479d-bebd-30aba7dee774 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:28.082258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 776/1541] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:28.082982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96770af6-7a22-479d-bebd-30aba7dee774 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:28.083828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a74ac-6ad8-42a3-8577-89ecc24dcab4 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:28.083958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96770af6-7a22-479d-bebd-30aba7dee774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:28.086928 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96770af6-7a22-479d-bebd-30aba7dee774 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:28.099993 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-465a74ac-6ad8-42a3-8577-89ecc24dcab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:28.130754 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 766/1542] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:27 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:28.132965 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96770af6-7a22-479d-bebd-30aba7dee774 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:28.136654 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-96770af6-7a22-479d-bebd-30aba7dee774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:28.181504 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:28.182534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:28.308478 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a74ac-6ad8-42a3-8577-89ecc24dcab4 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:28.319144 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70e58f-008a-49fd-b5c8-4b22f8dadd27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:28.325831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70e58f-008a-49fd-b5c8-4b22f8dadd27 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:28.325831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70e58f-008a-49fd-b5c8-4b22f8dadd27 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:28.325831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70e58f-008a-49fd-b5c8-4b22f8dadd27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:28.328092 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1f70e58f-008a-49fd-b5c8-4b22f8dadd27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:28.366417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96770af6-7a22-479d-bebd-30aba7dee774 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:28.387917 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985882c7-d6b2-4604-b8f3-f93392b23fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:28.389058 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985882c7-d6b2-4604-b8f3-f93392b23fcc service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:28.389553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985882c7-d6b2-4604-b8f3-f93392b23fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:28.390046 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985882c7-d6b2-4604-b8f3-f93392b23fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:28.435640 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985882c7-d6b2-4604-b8f3-f93392b23fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:28.438732 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985882c7-d6b2-4604-b8f3-f93392b23fcc service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:28.470471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70e58f-008a-49fd-b5c8-4b22f8dadd27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:28.473055 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70e58f-008a-49fd-b5c8-4b22f8dadd27 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:28.478704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 767/1543] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:28.480289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985882c7-d6b2-4604-b8f3-f93392b23fcc service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:28.503083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 777/1544] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:28.515937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985882c7-d6b2-4604-b8f3-f93392b23fcc service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:28.522270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2136b6-f714-461f-bc42-6e1cbf16da88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:28.523319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2136b6-f714-461f-bc42-6e1cbf16da88 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:28.523732 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2136b6-f714-461f-bc42-6e1cbf16da88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:28.525568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2136b6-f714-461f-bc42-6e1cbf16da88 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:28.525932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2136b6-f714-461f-bc42-6e1cbf16da88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:28.527826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2136b6-f714-461f-bc42-6e1cbf16da88 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:28.551788 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70e58f-008a-49fd-b5c8-4b22f8dadd27 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:28.563211 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 768/1545] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:28.567964 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70e58f-008a-49fd-b5c8-4b22f8dadd27 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:28.590007 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd06757-2433-4879-a462-799fd984deef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:28.591753 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd06757-2433-4879-a462-799fd984deef admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:28.592808 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd06757-2433-4879-a462-799fd984deef admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:28.594596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd06757-2433-4879-a462-799fd984deef admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:28.595444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd06757-2433-4879-a462-799fd984deef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:28.599360 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd06757-2433-4879-a462-799fd984deef admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:28.627165 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 778/1546] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:28.631391 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2136b6-f714-461f-bc42-6e1cbf16da88 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:28.646136 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2136b6-f714-461f-bc42-6e1cbf16da88 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:28.651535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476e99e-1b78-4caf-83b8-002b801754b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:28.652491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476e99e-1b78-4caf-83b8-002b801754b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:28.652783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476e99e-1b78-4caf-83b8-002b801754b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:28.655494 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476e99e-1b78-4caf-83b8-002b801754b5 admin admin] PATH_INFO: `/v3/projects/8d04a91ee39345a2abc01e0c988eaa1d/users/8aac939ef65c4805963f2917f1b77d0e/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:28.684504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476e99e-1b78-4caf-83b8-002b801754b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d04a91ee39345a2abc01e0c988eaa1d, user_id=8aac939ef65c4805963f2917f1b77d0e, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:28.688282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476e99e-1b78-4caf-83b8-002b801754b5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:28.721920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 769/1547] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:28 2022] PUT /identity/v3/projects/8d04a91ee39345a2abc01e0c988eaa1d/users/8aac939ef65c4805963f2917f1b77d0e/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:28.723279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c02c-7c24-4c18-b335-914782e37bb2 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:28.723790 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c02c-7c24-4c18-b335-914782e37bb2 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:28.724375 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013c02c-7c24-4c18-b335-914782e37bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:28.805040 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-e013c02c-7c24-4c18-b335-914782e37bb2 None None] MFA Rules not processed for user `8aac939ef65c4805963f2917f1b77d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:28.806927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e013c02c-7c24-4c18-b335-914782e37bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:29.007786 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 779/1548] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:29.033432 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476e99e-1b78-4caf-83b8-002b801754b5 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:29.177789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476e99e-1b78-4caf-83b8-002b801754b5 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:29.188460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad915d6-867c-4d78-b377-1f3f7d966a27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:29.192020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad915d6-867c-4d78-b377-1f3f7d966a27 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:29.192673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad915d6-867c-4d78-b377-1f3f7d966a27 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:29.194370 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad915d6-867c-4d78-b377-1f3f7d966a27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:29.229040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad915d6-867c-4d78-b377-1f3f7d966a27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:29.233516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad915d6-867c-4d78-b377-1f3f7d966a27 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:29.265211 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 770/1549] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:49:30.218209 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013c02c-7c24-4c18-b335-914782e37bb2 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:30.219849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e013c02c-7c24-4c18-b335-914782e37bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:30.302723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f30e7-879e-401d-8848-fcb3eb5226ea None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:30.303326 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f30e7-879e-401d-8848-fcb3eb5226ea None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:30.303326 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f30e7-879e-401d-8848-fcb3eb5226ea None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:30.304711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 771/1550] 10.222.0.36 () {54 vars in 921 bytes} [Sat Oct 15 13:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:49:30.372387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:30.387517 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013c02c-7c24-4c18-b335-914782e37bb2 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:30.395681 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:30.396524 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:30.396841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:30.397213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] PATH_INFO: `/v3/projects/8dd8bbe63f614ffcbfcfcaa67ce42d12` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:30.399123 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd8bbe63f614ffcbfcfcaa67ce42d12)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:30.401393 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:30.401846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc86ec-79e0-450c-ac9b-fc5c941cda96 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:30.402668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc86ec-79e0-450c-ac9b-fc5c941cda96 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:30.403441 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc86ec-79e0-450c-ac9b-fc5c941cda96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:30.467545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c3fc86ec-79e0-450c-ac9b-fc5c941cda96 None None] MFA Rules not processed for user `2e13692c1e83413a909d99bb7db2e4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:30.476294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c3fc86ec-79e0-450c-ac9b-fc5c941cda96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:30.637727 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd8bbe63f614ffcbfcfcaa67ce42d12', 'request_id': 'req-fe587503-8a06-48de-8aea-bed6dbf1d760'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:30.670481 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd8bbe63f614ffcbfcfcaa67ce42d12', 'request_id': 'req-fe587503-8a06-48de-8aea-bed6dbf1d760'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:30.737677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd8bbe63f614ffcbfcfcaa67ce42d12', 'request_id': 'req-fe587503-8a06-48de-8aea-bed6dbf1d760'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:30.806607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] The token cache is being invalidate because project 8dd8bbe63f614ffcbfcfcaa67ce42d12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:49:30.808428 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:30.809980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:30.815221 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 780/1551] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:49:30 2022] DELETE /identity/v3/projects/8dd8bbe63f614ffcbfcfcaa67ce42d12 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:30.822351 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:30.829857 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:30.872784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 772/1552] 10.222.0.36 () {58 vars in 1018 bytes} [Sat Oct 15 13:49:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:49:31.001103 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe587503-8a06-48de-8aea-bed6dbf1d760 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:31.011174 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04282e17-a3b0-49e0-a31b-1f9845dc4349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:31.011174 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04282e17-a3b0-49e0-a31b-1f9845dc4349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:31.011522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04282e17-a3b0-49e0-a31b-1f9845dc4349 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:31.012041 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04282e17-a3b0-49e0-a31b-1f9845dc4349 admin admin] PATH_INFO: `/v3/users/ed509a2359254f29bbfd5d09cc9ab06d` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:31.017898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04282e17-a3b0-49e0-a31b-1f9845dc4349 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed509a2359254f29bbfd5d09cc9ab06d)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:31.023780 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04282e17-a3b0-49e0-a31b-1f9845dc4349 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:31.146154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-04282e17-a3b0-49e0-a31b-1f9845dc4349 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed509a2359254f29bbfd5d09cc9ab06d'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:31.184100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-04282e17-a3b0-49e0-a31b-1f9845dc4349 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed509a2359254f29bbfd5d09cc9ab06d'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:31.188807 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-04282e17-a3b0-49e0-a31b-1f9845dc4349 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed509a2359254f29bbfd5d09cc9ab06d'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:31.189498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-04282e17-a3b0-49e0-a31b-1f9845dc4349 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed509a2359254f29bbfd5d09cc9ab06d'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:31.208701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-04282e17-a3b0-49e0-a31b-1f9845dc4349 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed509a2359254f29bbfd5d09cc9ab06d'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:31.267037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 781/1553] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:49:30 2022] DELETE /identity/v3/users/ed509a2359254f29bbfd5d09cc9ab06d => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:31.925941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc86ec-79e0-450c-ac9b-fc5c941cda96 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:31.932206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c3fc86ec-79e0-450c-ac9b-fc5c941cda96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:31.980429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc86ec-79e0-450c-ac9b-fc5c941cda96 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:32.033764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:32.103563 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437088a-3294-4b86-b18e-04d74091e3e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:32.104468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437088a-3294-4b86-b18e-04d74091e3e3 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:32.104718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437088a-3294-4b86-b18e-04d74091e3e3 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:32.106303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437088a-3294-4b86-b18e-04d74091e3e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:32.109158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6437088a-3294-4b86-b18e-04d74091e3e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:32.125962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437088a-3294-4b86-b18e-04d74091e3e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:32.128460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437088a-3294-4b86-b18e-04d74091e3e3 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:32.144482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04282e17-a3b0-49e0-a31b-1f9845dc4349 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:32.146726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-04282e17-a3b0-49e0-a31b-1f9845dc4349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:32.171842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:32.174460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 773/1554] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:32.355203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04282e17-a3b0-49e0-a31b-1f9845dc4349 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:32.362475 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3e5e0-b204-414d-a628-a228907daf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:32.363650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3e5e0-b204-414d-a628-a228907daf28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:32.365716 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3e5e0-b204-414d-a628-a228907daf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:32.366692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3e5e0-b204-414d-a628-a228907daf28 admin admin] PATH_INFO: `/v3/projects/3904810af867403b91a0e51f5e5c5e40` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:32.371971 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3e5e0-b204-414d-a628-a228907daf28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3904810af867403b91a0e51f5e5c5e40)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:32.379775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3e5e0-b204-414d-a628-a228907daf28 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:32.493651 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-8fd3e5e0-b204-414d-a628-a228907daf28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3904810af867403b91a0e51f5e5c5e40', 'request_id': 'req-8fd3e5e0-b204-414d-a628-a228907daf28'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:32.509259 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-8fd3e5e0-b204-414d-a628-a228907daf28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3904810af867403b91a0e51f5e5c5e40', 'request_id': 'req-8fd3e5e0-b204-414d-a628-a228907daf28'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:32.521895 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-8fd3e5e0-b204-414d-a628-a228907daf28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3904810af867403b91a0e51f5e5c5e40', 'request_id': 'req-8fd3e5e0-b204-414d-a628-a228907daf28'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:32.557410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-8fd3e5e0-b204-414d-a628-a228907daf28 admin admin] The token cache is being invalidate because project 3904810af867403b91a0e51f5e5c5e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:49:32.557831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-8fd3e5e0-b204-414d-a628-a228907daf28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:32.559052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-8fd3e5e0-b204-414d-a628-a228907daf28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:32.562367 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 782/1555] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:49:32 2022] DELETE /identity/v3/projects/3904810af867403b91a0e51f5e5c5e40 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:32.573191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734f7a3-b4f8-4a5b-92f5-d5bb8d5cdf35 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:32.573191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734f7a3-b4f8-4a5b-92f5-d5bb8d5cdf35 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:32.573539 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734f7a3-b4f8-4a5b-92f5-d5bb8d5cdf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:32.628374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f734f7a3-b4f8-4a5b-92f5-d5bb8d5cdf35 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:32.629743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f734f7a3-b4f8-4a5b-92f5-d5bb8d5cdf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:32.704806 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 774/1556] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:32.714418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59222-dbaa-4f23-bb58-6ff31cbb6f2e None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:32.715104 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59222-dbaa-4f23-bb58-6ff31cbb6f2e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:32.715648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59222-dbaa-4f23-bb58-6ff31cbb6f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:32.746141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-64c59222-dbaa-4f23-bb58-6ff31cbb6f2e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:32.748045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-64c59222-dbaa-4f23-bb58-6ff31cbb6f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:32.758629 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 783/1557] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:32.764330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734f7a3-b4f8-4a5b-92f5-d5bb8d5cdf35 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:32.780928 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734f7a3-b4f8-4a5b-92f5-d5bb8d5cdf35 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:32.786749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97057745-d6e6-4486-86fa-0666c217b8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:32.787393 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97057745-d6e6-4486-86fa-0666c217b8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:32.787669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97057745-d6e6-4486-86fa-0666c217b8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:32.787957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97057745-d6e6-4486-86fa-0666c217b8d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:32.788646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97057745-d6e6-4486-86fa-0666c217b8d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:49:32.788944 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97057745-d6e6-4486-86fa-0666c217b8d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:32.790572 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97057745-d6e6-4486-86fa-0666c217b8d3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:32.804552 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 775/1558] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:49:32.809671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c59222-dbaa-4f23-bb58-6ff31cbb6f2e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:32.820426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c59222-dbaa-4f23-bb58-6ff31cbb6f2e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:32.826525 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e5cd5-d3a1-43d7-9bd0-c665f1432640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:32.827169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e5cd5-d3a1-43d7-9bd0-c665f1432640 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:32.827444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e5cd5-d3a1-43d7-9bd0-c665f1432640 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:32.827709 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e5cd5-d3a1-43d7-9bd0-c665f1432640 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:32.828399 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e5cd5-d3a1-43d7-9bd0-c665f1432640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:32.830351 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e5cd5-d3a1-43d7-9bd0-c665f1432640 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:32.863495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 784/1559] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:32 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:32.870307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97057745-d6e6-4486-86fa-0666c217b8d3 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:32.881489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97057745-d6e6-4486-86fa-0666c217b8d3 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:32.982639 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b6508-3b7d-4422-b4fb-e15717d3d997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:32.983669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b6508-3b7d-4422-b4fb-e15717d3d997 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:32.983669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b6508-3b7d-4422-b4fb-e15717d3d997 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:32.983957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b6508-3b7d-4422-b4fb-e15717d3d997 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:32.984512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b6508-3b7d-4422-b4fb-e15717d3d997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:32.986734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b6508-3b7d-4422-b4fb-e15717d3d997 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:33.014823 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 776/1560] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:32 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:49:33.020272 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e5cd5-d3a1-43d7-9bd0-c665f1432640 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:33.038892 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e5cd5-d3a1-43d7-9bd0-c665f1432640 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:33.053229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5129e-af67-4180-b501-f55ea797fb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:33.054322 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5129e-af67-4180-b501-f55ea797fb24 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:33.055075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5129e-af67-4180-b501-f55ea797fb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:33.055334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5129e-af67-4180-b501-f55ea797fb24 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:33.056526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5129e-af67-4180-b501-f55ea797fb24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:33.058391 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5129e-af67-4180-b501-f55ea797fb24 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:33.087588 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 785/1561] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:33.090300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b6508-3b7d-4422-b4fb-e15717d3d997 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:33.111244 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b6508-3b7d-4422-b4fb-e15717d3d997 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:33.118557 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a095b-b8cf-41d9-b7c6-0fc4323e1be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:33.119709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a095b-b8cf-41d9-b7c6-0fc4323e1be3 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:33.120231 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a095b-b8cf-41d9-b7c6-0fc4323e1be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:33.120714 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a095b-b8cf-41d9-b7c6-0fc4323e1be3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:33.121906 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a095b-b8cf-41d9-b7c6-0fc4323e1be3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:33.125168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a095b-b8cf-41d9-b7c6-0fc4323e1be3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:33.147925 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 777/1562] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:33.154060 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5129e-af67-4180-b501-f55ea797fb24 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:33.170624 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5129e-af67-4180-b501-f55ea797fb24 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:33.183314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b3c33-a0e8-4ff5-ae15-ac3e21c0bf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:33.185320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b3c33-a0e8-4ff5-ae15-ac3e21c0bf17 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:33.186120 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b3c33-a0e8-4ff5-ae15-ac3e21c0bf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:33.186605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b3c33-a0e8-4ff5-ae15-ac3e21c0bf17 admin admin] PATH_INFO: `/v3/projects/7b3fde4da4504db1b9927e26b8eadb60/users/451ad89c9fa241ef86818b7bbe0f26de/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:33.217643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b3c33-a0e8-4ff5-ae15-ac3e21c0bf17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3fde4da4504db1b9927e26b8eadb60, user_id=451ad89c9fa241ef86818b7bbe0f26de, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:33.223673 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b3c33-a0e8-4ff5-ae15-ac3e21c0bf17 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:33.251271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 786/1563] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:33 2022] PUT /identity/v3/projects/7b3fde4da4504db1b9927e26b8eadb60/users/451ad89c9fa241ef86818b7bbe0f26de/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:33.268477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce3c1e-97d0-45ee-9d0f-a38f3432e9d8 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:33.269651 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce3c1e-97d0-45ee-9d0f-a38f3432e9d8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:33.270314 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce3c1e-97d0-45ee-9d0f-a38f3432e9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:33.338558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:33.345866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-66ce3c1e-97d0-45ee-9d0f-a38f3432e9d8 None None] MFA Rules not processed for user `451ad89c9fa241ef86818b7bbe0f26de`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:33.349331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-66ce3c1e-97d0-45ee-9d0f-a38f3432e9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:33.507563 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 778/1564] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:49:33.514401 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b3c33-a0e8-4ff5-ae15-ac3e21c0bf17 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:33.576423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:33.650036 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b3c33-a0e8-4ff5-ae15-ac3e21c0bf17 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:33.673618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811a271-3bce-47c5-a6b2-bebfedbb5293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:33.676089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811a271-3bce-47c5-a6b2-bebfedbb5293 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:33.679441 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811a271-3bce-47c5-a6b2-bebfedbb5293 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:33.680341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811a271-3bce-47c5-a6b2-bebfedbb5293 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:33.684040 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811a271-3bce-47c5-a6b2-bebfedbb5293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:33.698165 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811a271-3bce-47c5-a6b2-bebfedbb5293 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:33.734011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce3c1e-97d0-45ee-9d0f-a38f3432e9d8 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:33.736827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-66ce3c1e-97d0-45ee-9d0f-a38f3432e9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:33.748218 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 787/1565] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:33 2022] POST /identity/v3/projects => generated 377 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:33.752616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811a271-3bce-47c5-a6b2-bebfedbb5293 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:33.764492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce3c1e-97d0-45ee-9d0f-a38f3432e9d8 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:33.961318 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811a271-3bce-47c5-a6b2-bebfedbb5293 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:33.990352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a336ad9-bb64-4ab9-bd1e-8080cb9d6a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:33.991434 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a336ad9-bb64-4ab9-bd1e-8080cb9d6a07 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:33.991958 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a336ad9-bb64-4ab9-bd1e-8080cb9d6a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:33.992450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a336ad9-bb64-4ab9-bd1e-8080cb9d6a07 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:33.994692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a336ad9-bb64-4ab9-bd1e-8080cb9d6a07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:34.008429 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a336ad9-bb64-4ab9-bd1e-8080cb9d6a07 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:34.010878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee19a2b-5fd8-4c58-9ae1-29786640d30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:34.012528 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee19a2b-5fd8-4c58-9ae1-29786640d30a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:34.014340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee19a2b-5fd8-4c58-9ae1-29786640d30a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:34.015096 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee19a2b-5fd8-4c58-9ae1-29786640d30a admin admin] PATH_INFO: `/v3/users/8aac939ef65c4805963f2917f1b77d0e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:34.023413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee19a2b-5fd8-4c58-9ae1-29786640d30a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aac939ef65c4805963f2917f1b77d0e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:34.040293 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee19a2b-5fd8-4c58-9ae1-29786640d30a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:34.057189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 788/1566] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:33 2022] POST /identity/v3/users => generated 295 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:34.061467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a336ad9-bb64-4ab9-bd1e-8080cb9d6a07 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:34.094204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a336ad9-bb64-4ab9-bd1e-8080cb9d6a07 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:34.112482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24d1e9-2bc8-4111-846b-1c0d9ed60d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:34.113992 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24d1e9-2bc8-4111-846b-1c0d9ed60d21 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:34.114457 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24d1e9-2bc8-4111-846b-1c0d9ed60d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:34.115421 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24d1e9-2bc8-4111-846b-1c0d9ed60d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:34.116240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24d1e9-2bc8-4111-846b-1c0d9ed60d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:34.119913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24d1e9-2bc8-4111-846b-1c0d9ed60d21 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:34.159325 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 789/1567] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:49:34.160826 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24d1e9-2bc8-4111-846b-1c0d9ed60d21 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:34.176223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24d1e9-2bc8-4111-846b-1c0d9ed60d21 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:34.201934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289405a8-4341-4f07-9ac5-4a99510ca19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:34.203087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289405a8-4341-4f07-9ac5-4a99510ca19f admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:34.203579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289405a8-4341-4f07-9ac5-4a99510ca19f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:34.204178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289405a8-4341-4f07-9ac5-4a99510ca19f admin admin] PATH_INFO: `/v3/projects/406b33a302174fdaae923c02ec44d29a/users/c5609fb7ffcf42d9b386763ea10a789c/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:34.233523 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289405a8-4341-4f07-9ac5-4a99510ca19f admin admin] RBAC: Authorizing `identity:create_grant(project_id=406b33a302174fdaae923c02ec44d29a, user_id=c5609fb7ffcf42d9b386763ea10a789c, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:34.238442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289405a8-4341-4f07-9ac5-4a99510ca19f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:34.247406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4ee19a2b-5fd8-4c58-9ae1-29786640d30a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aac939ef65c4805963f2917f1b77d0e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:34.250592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4ee19a2b-5fd8-4c58-9ae1-29786640d30a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aac939ef65c4805963f2917f1b77d0e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:34.253609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4ee19a2b-5fd8-4c58-9ae1-29786640d30a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aac939ef65c4805963f2917f1b77d0e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:34.260817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 790/1568] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:34 2022] PUT /identity/v3/projects/406b33a302174fdaae923c02ec44d29a/users/c5609fb7ffcf42d9b386763ea10a789c/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:34.269315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289405a8-4341-4f07-9ac5-4a99510ca19f admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:34.272624 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-289405a8-4341-4f07-9ac5-4a99510ca19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:34.288514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4ee19a2b-5fd8-4c58-9ae1-29786640d30a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aac939ef65c4805963f2917f1b77d0e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:34.309802 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4ee19a2b-5fd8-4c58-9ae1-29786640d30a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aac939ef65c4805963f2917f1b77d0e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:34.395330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 779/1569] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:49:33 2022] DELETE /identity/v3/users/8aac939ef65c4805963f2917f1b77d0e => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:34.402681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007f556-e621-4ed4-b327-3c14845bcaf4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:34.403139 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007f556-e621-4ed4-b327-3c14845bcaf4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:34.403657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007f556-e621-4ed4-b327-3c14845bcaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:34.424417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289405a8-4341-4f07-9ac5-4a99510ca19f admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:34.458537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:34.474260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7007f556-e621-4ed4-b327-3c14845bcaf4 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:34.476594 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7007f556-e621-4ed4-b327-3c14845bcaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:34.532932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:34.587248 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136daa91-f8b3-4a32-a06f-09072d1ba019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:34.588301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136daa91-f8b3-4a32-a06f-09072d1ba019 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:34.589324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136daa91-f8b3-4a32-a06f-09072d1ba019 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:34.589996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136daa91-f8b3-4a32-a06f-09072d1ba019 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:34.591401 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136daa91-f8b3-4a32-a06f-09072d1ba019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:34.594601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136daa91-f8b3-4a32-a06f-09072d1ba019 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:34.620077 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 791/1570] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:49:34.624623 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136daa91-f8b3-4a32-a06f-09072d1ba019 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:34.642220 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136daa91-f8b3-4a32-a06f-09072d1ba019 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:34.648462 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a350b60-a833-4026-ad62-379287847c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:34.649176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a350b60-a833-4026-ad62-379287847c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:34.649508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a350b60-a833-4026-ad62-379287847c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:34.649792 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a350b60-a833-4026-ad62-379287847c7e admin admin] PATH_INFO: `/v3/projects/406b33a302174fdaae923c02ec44d29a/users/c5609fb7ffcf42d9b386763ea10a789c/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:34.652290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a350b60-a833-4026-ad62-379287847c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=406b33a302174fdaae923c02ec44d29a, user_id=c5609fb7ffcf42d9b386763ea10a789c, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:34.655275 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a350b60-a833-4026-ad62-379287847c7e admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:34.669840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 792/1571] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:34 2022] PUT /identity/v3/projects/406b33a302174fdaae923c02ec44d29a/users/c5609fb7ffcf42d9b386763ea10a789c/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:34.675807 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 780/1572] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:49:34.687602 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d466b51-5d29-490e-8517-6923ad2e27ab None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:34.688049 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d466b51-5d29-490e-8517-6923ad2e27ab None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:34.688199 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d466b51-5d29-490e-8517-6923ad2e27ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:34.694610 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007f556-e621-4ed4-b327-3c14845bcaf4 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:34.698589 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7007f556-e621-4ed4-b327-3c14845bcaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:34.782464 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-1d466b51-5d29-490e-8517-6923ad2e27ab None None] MFA Rules not processed for user `c5609fb7ffcf42d9b386763ea10a789c`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:34.787367 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1d466b51-5d29-490e-8517-6923ad2e27ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:34.810522 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007f556-e621-4ed4-b327-3c14845bcaf4 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:34.952567 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 793/1573] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:34.952986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bade8a-0dec-40d3-b2ce-17dbd0dab1d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:34.954070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bade8a-0dec-40d3-b2ce-17dbd0dab1d6 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:34.954579 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bade8a-0dec-40d3-b2ce-17dbd0dab1d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:34.955647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bade8a-0dec-40d3-b2ce-17dbd0dab1d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:34.975087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d466b51-5d29-490e-8517-6923ad2e27ab None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:34.979096 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1d466b51-5d29-490e-8517-6923ad2e27ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:35.054129 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bade8a-0dec-40d3-b2ce-17dbd0dab1d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:35.056508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bade8a-0dec-40d3-b2ce-17dbd0dab1d6 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:35.081562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 781/1574] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:35.164566 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d466b51-5d29-490e-8517-6923ad2e27ab None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:35.176563 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40744bd2-bef4-4c05-85fc-65ac458157d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:35.177368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40744bd2-bef4-4c05-85fc-65ac458157d7 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:35.177806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40744bd2-bef4-4c05-85fc-65ac458157d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:35.178094 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40744bd2-bef4-4c05-85fc-65ac458157d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:35.182853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-40744bd2-bef4-4c05-85fc-65ac458157d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:35.317708 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:35.354760 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40744bd2-bef4-4c05-85fc-65ac458157d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:35.356529 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40744bd2-bef4-4c05-85fc-65ac458157d7 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:35.371111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 794/1575] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:36.328978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bade8a-0dec-40d3-b2ce-17dbd0dab1d6 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:36.332178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b6bade8a-0dec-40d3-b2ce-17dbd0dab1d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:36.357023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bade8a-0dec-40d3-b2ce-17dbd0dab1d6 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:36.376862 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c10943-83c5-4870-85d0-a139a2eb00bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:36.377671 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c10943-83c5-4870-85d0-a139a2eb00bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:36.378016 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c10943-83c5-4870-85d0-a139a2eb00bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:36.378356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c10943-83c5-4870-85d0-a139a2eb00bb admin admin] PATH_INFO: `/v3/projects/8d04a91ee39345a2abc01e0c988eaa1d` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:36.382751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c10943-83c5-4870-85d0-a139a2eb00bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d04a91ee39345a2abc01e0c988eaa1d)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:36.385011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c10943-83c5-4870-85d0-a139a2eb00bb admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:36.429755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:36.472821 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-45c10943-83c5-4870-85d0-a139a2eb00bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d04a91ee39345a2abc01e0c988eaa1d', 'request_id': 'req-45c10943-83c5-4870-85d0-a139a2eb00bb'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:36.485472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-45c10943-83c5-4870-85d0-a139a2eb00bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d04a91ee39345a2abc01e0c988eaa1d', 'request_id': 'req-45c10943-83c5-4870-85d0-a139a2eb00bb'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:36.500800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-45c10943-83c5-4870-85d0-a139a2eb00bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d04a91ee39345a2abc01e0c988eaa1d', 'request_id': 'req-45c10943-83c5-4870-85d0-a139a2eb00bb'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:36.530405 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-45c10943-83c5-4870-85d0-a139a2eb00bb admin admin] The token cache is being invalidate because project 8d04a91ee39345a2abc01e0c988eaa1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:49:36.530708 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-45c10943-83c5-4870-85d0-a139a2eb00bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:36.531449 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-45c10943-83c5-4870-85d0-a139a2eb00bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:36.535563 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 782/1576] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:49:36 2022] DELETE /identity/v3/projects/8d04a91ee39345a2abc01e0c988eaa1d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:36.540885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e3dfc-4677-4d3a-989d-0a406f071598 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:36.541741 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e3dfc-4677-4d3a-989d-0a406f071598 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:36.541741 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e3dfc-4677-4d3a-989d-0a406f071598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:36.567280 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-994e3dfc-4677-4d3a-989d-0a406f071598 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:36.568817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-994e3dfc-4677-4d3a-989d-0a406f071598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:36.661776 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 795/1577] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:36.670441 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc107ea0-baf6-437c-bedb-2de867964a1f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:36.670728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc107ea0-baf6-437c-bedb-2de867964a1f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:36.671004 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc107ea0-baf6-437c-bedb-2de867964a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:36.704553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-cc107ea0-baf6-437c-bedb-2de867964a1f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:36.705983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cc107ea0-baf6-437c-bedb-2de867964a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:36.714851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 783/1578] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:36.719839 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e3dfc-4677-4d3a-989d-0a406f071598 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:36.732136 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e3dfc-4677-4d3a-989d-0a406f071598 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:36.738046 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe79cc7-f603-4fc2-9b49-af7185280df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:36.738732 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe79cc7-f603-4fc2-9b49-af7185280df0 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:36.739006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe79cc7-f603-4fc2-9b49-af7185280df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:36.739272 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe79cc7-f603-4fc2-9b49-af7185280df0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:36.739999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe79cc7-f603-4fc2-9b49-af7185280df0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:49:36.740300 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe79cc7-f603-4fc2-9b49-af7185280df0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:36.742055 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe79cc7-f603-4fc2-9b49-af7185280df0 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:36.764781 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 796/1579] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:49:36.766333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc107ea0-baf6-437c-bedb-2de867964a1f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:36.776673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc107ea0-baf6-437c-bedb-2de867964a1f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:36.782451 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a320fb-9d2d-494b-8e8a-56079846bc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:36.783107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a320fb-9d2d-494b-8e8a-56079846bc50 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:36.783384 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a320fb-9d2d-494b-8e8a-56079846bc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:36.783654 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a320fb-9d2d-494b-8e8a-56079846bc50 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:36.784358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a320fb-9d2d-494b-8e8a-56079846bc50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:36.786388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a320fb-9d2d-494b-8e8a-56079846bc50 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:36.820075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 784/1580] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:36 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:36.824847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe79cc7-f603-4fc2-9b49-af7185280df0 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:36.837550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe79cc7-f603-4fc2-9b49-af7185280df0 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:36.932844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157608de-1402-422c-a119-a17f81fbacc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:36.933781 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157608de-1402-422c-a119-a17f81fbacc1 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:36.937316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157608de-1402-422c-a119-a17f81fbacc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:36.937316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157608de-1402-422c-a119-a17f81fbacc1 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:36.937316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157608de-1402-422c-a119-a17f81fbacc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:36.938868 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157608de-1402-422c-a119-a17f81fbacc1 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:36.981439 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 797/1581] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:36 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:36.986085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a320fb-9d2d-494b-8e8a-56079846bc50 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:37.015988 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a320fb-9d2d-494b-8e8a-56079846bc50 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:37.024296 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb5543-7421-4ba3-888c-fafb01959ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:37.025290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb5543-7421-4ba3-888c-fafb01959ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:37.025747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb5543-7421-4ba3-888c-fafb01959ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:37.026049 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb5543-7421-4ba3-888c-fafb01959ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:37.026683 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb5543-7421-4ba3-888c-fafb01959ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:37.031601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb5543-7421-4ba3-888c-fafb01959ea9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:37.068660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 785/1582] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:37.083124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157608de-1402-422c-a119-a17f81fbacc1 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:37.097011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157608de-1402-422c-a119-a17f81fbacc1 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:37.127909 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1feba5b-073f-42c6-b3c5-8ed93b8603dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:37.128910 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feba5b-073f-42c6-b3c5-8ed93b8603dc admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:37.129319 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feba5b-073f-42c6-b3c5-8ed93b8603dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:37.131017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feba5b-073f-42c6-b3c5-8ed93b8603dc admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:37.134015 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1feba5b-073f-42c6-b3c5-8ed93b8603dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:37.137905 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1feba5b-073f-42c6-b3c5-8ed93b8603dc admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:37.192783 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 798/1583] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:37.201549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb5543-7421-4ba3-888c-fafb01959ea9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:37.236381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb5543-7421-4ba3-888c-fafb01959ea9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:37.251723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4875994-4098-4f8a-a323-1b970a0db0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:37.252681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4875994-4098-4f8a-a323-1b970a0db0a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:37.253050 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4875994-4098-4f8a-a323-1b970a0db0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:37.257072 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4875994-4098-4f8a-a323-1b970a0db0a1 admin admin] PATH_INFO: `/v3/projects/9c0fca1ba6844ea0b6f3a3ea311b98b1/users/06c580268e0f4f2b85984c3c960a9117/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:37.304859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4875994-4098-4f8a-a323-1b970a0db0a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c0fca1ba6844ea0b6f3a3ea311b98b1, user_id=06c580268e0f4f2b85984c3c960a9117, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:37.312507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4875994-4098-4f8a-a323-1b970a0db0a1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:37.333963 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 786/1584] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:37 2022] PUT /identity/v3/projects/9c0fca1ba6844ea0b6f3a3ea311b98b1/users/06c580268e0f4f2b85984c3c960a9117/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:37.344507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f036c7-c01c-47f5-a889-d8bb49e55166 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:37.344883 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f036c7-c01c-47f5-a889-d8bb49e55166 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:37.345080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f036c7-c01c-47f5-a889-d8bb49e55166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:37.404074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-c1f036c7-c01c-47f5-a889-d8bb49e55166 None None] MFA Rules not processed for user `06c580268e0f4f2b85984c3c960a9117`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:37.409576 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c1f036c7-c01c-47f5-a889-d8bb49e55166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:37.485095 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:37.557285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4875994-4098-4f8a-a323-1b970a0db0a1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:37.558342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 799/1585] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:37.733266 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4875994-4098-4f8a-a323-1b970a0db0a1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:37.738558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7ab08-942d-4da7-bd9d-330b42846df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:37.749752 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7ab08-942d-4da7-bd9d-330b42846df1 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:37.751972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7ab08-942d-4da7-bd9d-330b42846df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:37.752284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7ab08-942d-4da7-bd9d-330b42846df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:37.754541 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7ab08-942d-4da7-bd9d-330b42846df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:37.759970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7ab08-942d-4da7-bd9d-330b42846df1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:37.845890 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 787/1586] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:37 2022] POST /identity/v3/projects => generated 383 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:49:37.857719 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f036c7-c01c-47f5-a889-d8bb49e55166 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:37.895991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f036c7-c01c-47f5-a889-d8bb49e55166 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:38.016978 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfddb0c-0d0d-4533-9a9e-f6fd4bb9621c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:38.018050 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfddb0c-0d0d-4533-9a9e-f6fd4bb9621c admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:38.018693 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfddb0c-0d0d-4533-9a9e-f6fd4bb9621c admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:38.019107 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfddb0c-0d0d-4533-9a9e-f6fd4bb9621c admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:38.020529 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfddb0c-0d0d-4533-9a9e-f6fd4bb9621c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:38.025529 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfddb0c-0d0d-4533-9a9e-f6fd4bb9621c admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:38.067943 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 800/1587] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:37 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:38.071925 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7ab08-942d-4da7-bd9d-330b42846df1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:38.099425 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7ab08-942d-4da7-bd9d-330b42846df1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:38.114310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a19198-f184-4d71-a22b-5fc9133f1519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:38.118187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a19198-f184-4d71-a22b-5fc9133f1519 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:38.118596 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a19198-f184-4d71-a22b-5fc9133f1519 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:38.118943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a19198-f184-4d71-a22b-5fc9133f1519 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:38.120680 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a19198-f184-4d71-a22b-5fc9133f1519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:38.122873 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a19198-f184-4d71-a22b-5fc9133f1519 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:38.140902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 788/1588] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:49:38.145456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfddb0c-0d0d-4533-9a9e-f6fd4bb9621c admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:38.157978 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfddb0c-0d0d-4533-9a9e-f6fd4bb9621c admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:38.163033 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab41390-5ba4-4e42-9f6a-91da49af818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:38.163620 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab41390-5ba4-4e42-9f6a-91da49af818d admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:38.163816 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab41390-5ba4-4e42-9f6a-91da49af818d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:38.164138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab41390-5ba4-4e42-9f6a-91da49af818d admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:38.164821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab41390-5ba4-4e42-9f6a-91da49af818d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:38.166505 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab41390-5ba4-4e42-9f6a-91da49af818d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:38.179644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 801/1589] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:49:38.184524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a19198-f184-4d71-a22b-5fc9133f1519 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:38.204262 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a19198-f184-4d71-a22b-5fc9133f1519 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:38.212347 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61895232-bd64-4313-b77b-ca3ed00c39bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:38.213638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61895232-bd64-4313-b77b-ca3ed00c39bc admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:38.214117 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61895232-bd64-4313-b77b-ca3ed00c39bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:38.214577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61895232-bd64-4313-b77b-ca3ed00c39bc admin admin] PATH_INFO: `/v3/projects/dcea1503d9354b29b8cb7af672c41470/users/a202b55ac8244d8cac6cb3821673e7c0/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:38.239266 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61895232-bd64-4313-b77b-ca3ed00c39bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcea1503d9354b29b8cb7af672c41470, user_id=a202b55ac8244d8cac6cb3821673e7c0, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:38.242252 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61895232-bd64-4313-b77b-ca3ed00c39bc admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:38.256072 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:38.265616 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 789/1590] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:38 2022] PUT /identity/v3/projects/dcea1503d9354b29b8cb7af672c41470/users/a202b55ac8244d8cac6cb3821673e7c0/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:38.279380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ea577-427e-41fd-9bea-5e28f0a22ef7 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:38.279672 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ea577-427e-41fd-9bea-5e28f0a22ef7 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:38.280025 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ea577-427e-41fd-9bea-5e28f0a22ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:38.322595 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-822ea577-427e-41fd-9bea-5e28f0a22ef7 None None] MFA Rules not processed for user `a202b55ac8244d8cac6cb3821673e7c0`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:38.324346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-822ea577-427e-41fd-9bea-5e28f0a22ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:38.415160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 802/1591] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:38.419777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61895232-bd64-4313-b77b-ca3ed00c39bc admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:38.501665 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61895232-bd64-4313-b77b-ca3ed00c39bc admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:38.508820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720f25f-b231-4f48-9b4a-9e6276ad3ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:38.509591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720f25f-b231-4f48-9b4a-9e6276ad3ecd admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:38.510104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720f25f-b231-4f48-9b4a-9e6276ad3ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:38.512874 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720f25f-b231-4f48-9b4a-9e6276ad3ecd admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:38.516049 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720f25f-b231-4f48-9b4a-9e6276ad3ecd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:38.522576 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720f25f-b231-4f48-9b4a-9e6276ad3ecd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:38.586713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 790/1592] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:49:38 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:38.594271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ea577-427e-41fd-9bea-5e28f0a22ef7 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:38.615801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ea577-427e-41fd-9bea-5e28f0a22ef7 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:38.697507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:38.730544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99aae4-4986-41de-b5e4-cebb5ae9b30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:38.731112 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99aae4-4986-41de-b5e4-cebb5ae9b30d admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:38.731587 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99aae4-4986-41de-b5e4-cebb5ae9b30d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:38.736540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99aae4-4986-41de-b5e4-cebb5ae9b30d admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:38.737915 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99aae4-4986-41de-b5e4-cebb5ae9b30d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:38.742362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99aae4-4986-41de-b5e4-cebb5ae9b30d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:38.781183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 803/1593] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:49:38 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:49:38.785862 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720f25f-b231-4f48-9b4a-9e6276ad3ecd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:38.808759 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720f25f-b231-4f48-9b4a-9e6276ad3ecd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:38.823216 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d227dd-13f2-495d-9d47-d8c49741c4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:38.824257 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d227dd-13f2-495d-9d47-d8c49741c4af admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:38.824700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d227dd-13f2-495d-9d47-d8c49741c4af admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:38.825155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d227dd-13f2-495d-9d47-d8c49741c4af admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:38.826352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d227dd-13f2-495d-9d47-d8c49741c4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:38.829296 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d227dd-13f2-495d-9d47-d8c49741c4af admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:38.868796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 791/1594] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:38.873061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99aae4-4986-41de-b5e4-cebb5ae9b30d admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:38.908241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99aae4-4986-41de-b5e4-cebb5ae9b30d admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:38.908241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75227c6f-3e72-4b8f-9a98-7a01ac393bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:38.908241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75227c6f-3e72-4b8f-9a98-7a01ac393bfe admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:38.908241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75227c6f-3e72-4b8f-9a98-7a01ac393bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:38.909268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75227c6f-3e72-4b8f-9a98-7a01ac393bfe admin admin] PATH_INFO: `/v3/projects/a1e5848dda374b96a551bd65c5df01c5/users/c065b783c56843e7b9648f859bce2ec1/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:38.937773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75227c6f-3e72-4b8f-9a98-7a01ac393bfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1e5848dda374b96a551bd65c5df01c5, user_id=c065b783c56843e7b9648f859bce2ec1, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:38.940569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75227c6f-3e72-4b8f-9a98-7a01ac393bfe admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:38.965368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 804/1595] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:38 2022] PUT /identity/v3/projects/a1e5848dda374b96a551bd65c5df01c5/users/c065b783c56843e7b9648f859bce2ec1/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:49:38.970974 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d227dd-13f2-495d-9d47-d8c49741c4af admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:38.987138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d227dd-13f2-495d-9d47-d8c49741c4af admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:39.109398 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289090c6-6f2c-4c75-85ac-1869382dfced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:39.110455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289090c6-6f2c-4c75-85ac-1869382dfced admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:39.111028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289090c6-6f2c-4c75-85ac-1869382dfced admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:39.111502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289090c6-6f2c-4c75-85ac-1869382dfced admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:39.112443 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289090c6-6f2c-4c75-85ac-1869382dfced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:39.114703 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289090c6-6f2c-4c75-85ac-1869382dfced admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:39.129365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 792/1596] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:49:39.134223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75227c6f-3e72-4b8f-9a98-7a01ac393bfe admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:39.158140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75227c6f-3e72-4b8f-9a98-7a01ac393bfe admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:39.167344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a304e-b970-4b1f-b27f-45a0f9c6ca29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:39.168223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a304e-b970-4b1f-b27f-45a0f9c6ca29 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:39.168574 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a304e-b970-4b1f-b27f-45a0f9c6ca29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:39.168925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a304e-b970-4b1f-b27f-45a0f9c6ca29 admin admin] PATH_INFO: `/v3/projects/a1e5848dda374b96a551bd65c5df01c5/users/c065b783c56843e7b9648f859bce2ec1/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:39.178489 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a304e-b970-4b1f-b27f-45a0f9c6ca29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1e5848dda374b96a551bd65c5df01c5, user_id=c065b783c56843e7b9648f859bce2ec1, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:39.186145 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a304e-b970-4b1f-b27f-45a0f9c6ca29 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:39.199359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 805/1597] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:49:39 2022] PUT /identity/v3/projects/a1e5848dda374b96a551bd65c5df01c5/users/c065b783c56843e7b9648f859bce2ec1/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:39.211527 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa014700-008d-4d04-888b-bf36ff171a64 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:39.211808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa014700-008d-4d04-888b-bf36ff171a64 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:39.212121 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa014700-008d-4d04-888b-bf36ff171a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:39.243538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-fa014700-008d-4d04-888b-bf36ff171a64 None None] MFA Rules not processed for user `c065b783c56843e7b9648f859bce2ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:39.245050 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fa014700-008d-4d04-888b-bf36ff171a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:39.317989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:39.337940 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 793/1598] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:39.344694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a304e-b970-4b1f-b27f-45a0f9c6ca29 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:39.346545 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-fd1a304e-b970-4b1f-b27f-45a0f9c6ca29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:39.422467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a304e-b970-4b1f-b27f-45a0f9c6ca29 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:39.426848 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f1821-5a0f-41fe-be60-ab024d27aa45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:39.427557 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f1821-5a0f-41fe-be60-ab024d27aa45 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:39.427833 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f1821-5a0f-41fe-be60-ab024d27aa45 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:39.428162 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f1821-5a0f-41fe-be60-ab024d27aa45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:39.507359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f1821-5a0f-41fe-be60-ab024d27aa45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:39.509076 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f1821-5a0f-41fe-be60-ab024d27aa45 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:39.520330 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 806/1599] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:43.063347 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa014700-008d-4d04-888b-bf36ff171a64 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:43.074858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa014700-008d-4d04-888b-bf36ff171a64 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:43.079775 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f89f4-0fdb-4cb1-aa10-7e514ff10d29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:43.080477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f89f4-0fdb-4cb1-aa10-7e514ff10d29 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:43.080767 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f89f4-0fdb-4cb1-aa10-7e514ff10d29 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:43.081061 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f89f4-0fdb-4cb1-aa10-7e514ff10d29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:43.083404 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-246f89f4-0fdb-4cb1-aa10-7e514ff10d29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:43.208112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f89f4-0fdb-4cb1-aa10-7e514ff10d29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:43.211274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f89f4-0fdb-4cb1-aa10-7e514ff10d29 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:43.232669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 794/1600] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:49:44.808181 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f1821-5a0f-41fe-be60-ab024d27aa45 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:44.837839 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f1821-5a0f-41fe-be60-ab024d27aa45 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:44.853640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384f493-5013-4f58-ae83-6cd7aac063ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:44.854458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384f493-5013-4f58-ae83-6cd7aac063ca service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:44.854740 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384f493-5013-4f58-ae83-6cd7aac063ca service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:44.855041 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384f493-5013-4f58-ae83-6cd7aac063ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:44.934403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:44.948213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384f493-5013-4f58-ae83-6cd7aac063ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:44.949936 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384f493-5013-4f58-ae83-6cd7aac063ca service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:44.962881 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 807/1601] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:49:48.450319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f89f4-0fdb-4cb1-aa10-7e514ff10d29 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:48.466564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f89f4-0fdb-4cb1-aa10-7e514ff10d29 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:48.472607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad905d3-e744-4de5-81c7-e74d6a423e26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:48.473887 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad905d3-e744-4de5-81c7-e74d6a423e26 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:48.474530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad905d3-e744-4de5-81c7-e74d6a423e26 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:48.475200 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad905d3-e744-4de5-81c7-e74d6a423e26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:48.493125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad905d3-e744-4de5-81c7-e74d6a423e26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:48.496031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad905d3-e744-4de5-81c7-e74d6a423e26 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:48.515958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 795/1602] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:54.955820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=5 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:57.619293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384f493-5013-4f58-ae83-6cd7aac063ca service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:57.713827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384f493-5013-4f58-ae83-6cd7aac063ca service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:57.719559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1036a-5c3a-4301-83ac-99b3cd500713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:57.720415 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1036a-5c3a-4301-83ac-99b3cd500713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:57.720709 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1036a-5c3a-4301-83ac-99b3cd500713 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:57.721080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1036a-5c3a-4301-83ac-99b3cd500713 admin admin] PATH_INFO: `/v3/users/06c580268e0f4f2b85984c3c960a9117` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:57.722468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1036a-5c3a-4301-83ac-99b3cd500713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06c580268e0f4f2b85984c3c960a9117)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:57.724501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1036a-5c3a-4301-83ac-99b3cd500713 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:57.792154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7da1036a-5c3a-4301-83ac-99b3cd500713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06c580268e0f4f2b85984c3c960a9117'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:57.808340 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7da1036a-5c3a-4301-83ac-99b3cd500713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06c580268e0f4f2b85984c3c960a9117'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:57.810448 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7da1036a-5c3a-4301-83ac-99b3cd500713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06c580268e0f4f2b85984c3c960a9117'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:57.811715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7da1036a-5c3a-4301-83ac-99b3cd500713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06c580268e0f4f2b85984c3c960a9117'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:57.825228 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7da1036a-5c3a-4301-83ac-99b3cd500713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06c580268e0f4f2b85984c3c960a9117'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:57.855749 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 808/1603] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:49:57 2022] DELETE /identity/v3/users/06c580268e0f4f2b85984c3c960a9117 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:57.860979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830eadf3-030d-45ff-bd22-f76c19e53036 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:57.861860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830eadf3-030d-45ff-bd22-f76c19e53036 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:57.861860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830eadf3-030d-45ff-bd22-f76c19e53036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:57.888508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-830eadf3-030d-45ff-bd22-f76c19e53036 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:57.890248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-830eadf3-030d-45ff-bd22-f76c19e53036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:57.992514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 796/1604] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:58.007133 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1036a-5c3a-4301-83ac-99b3cd500713 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:58.009947 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7da1036a-5c3a-4301-83ac-99b3cd500713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:58.104263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1036a-5c3a-4301-83ac-99b3cd500713 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:58.109574 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978422d-6ba3-4c65-aff5-cf5728863757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:58.110429 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978422d-6ba3-4c65-aff5-cf5728863757 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:58.110801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978422d-6ba3-4c65-aff5-cf5728863757 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:58.111233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978422d-6ba3-4c65-aff5-cf5728863757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:58.124392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978422d-6ba3-4c65-aff5-cf5728863757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:58.128537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978422d-6ba3-4c65-aff5-cf5728863757 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:58.158325 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 809/1605] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:58.266076 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dde606-a5f4-454c-878a-a544a0d13924 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:58.266970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dde606-a5f4-454c-878a-a544a0d13924 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:58.267488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dde606-a5f4-454c-878a-a544a0d13924 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:58.272228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 797/1606] 10.222.0.36 () {54 vars in 921 bytes} [Sat Oct 15 13:49:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:49:58.285988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad81e7-ccc6-4918-ba3f-03d1d85947d4 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:58.287141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad81e7-ccc6-4918-ba3f-03d1d85947d4 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:58.287141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad81e7-ccc6-4918-ba3f-03d1d85947d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:58.334982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-29ad81e7-ccc6-4918-ba3f-03d1d85947d4 None None] MFA Rules not processed for user `1c6720518c03413aa8f85e4d6142fe87`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:58.336295 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-29ad81e7-ccc6-4918-ba3f-03d1d85947d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:58.394196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:58.518756 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 810/1607] 10.222.0.36 () {58 vars in 1018 bytes} [Sat Oct 15 13:49:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:49:58.522219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dde606-a5f4-454c-878a-a544a0d13924 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:58.526377 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a8dde606-a5f4-454c-878a-a544a0d13924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:58.557667 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a587acf-c21a-44c5-832f-35b5952afcfd None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:58.557964 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a587acf-c21a-44c5-832f-35b5952afcfd None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:58.563732 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a587acf-c21a-44c5-832f-35b5952afcfd None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:58.567348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 811/1608] 10.222.0.36 () {54 vars in 961 bytes} [Sat Oct 15 13:49:58 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 15 13:49:58.576494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37151e-78b5-42c0-80c8-3aa3050bb6b4 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:58.576825 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37151e-78b5-42c0-80c8-3aa3050bb6b4 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:58.577124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37151e-78b5-42c0-80c8-3aa3050bb6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:58.602001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dde606-a5f4-454c-878a-a544a0d13924 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:58.614227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-8e37151e-78b5-42c0-80c8-3aa3050bb6b4 None None] MFA Rules not processed for user `1c6720518c03413aa8f85e4d6142fe87`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:58.615654 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8e37151e-78b5-42c0-80c8-3aa3050bb6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:58.616904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc4c61-c5fc-4d9b-9871-b93d610f7013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:58.618061 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4c61-c5fc-4d9b-9871-b93d610f7013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:58.618524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4c61-c5fc-4d9b-9871-b93d610f7013 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:58.618995 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4c61-c5fc-4d9b-9871-b93d610f7013 admin admin] PATH_INFO: `/v3/users/0692c115a2264edda61b9373bf1b7cf6` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:58.631414 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 812/1609] 10.222.0.36 () {58 vars in 1058 bytes} [Sat Oct 15 13:49:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:58.632924 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc4c61-c5fc-4d9b-9871-b93d610f7013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0692c115a2264edda61b9373bf1b7cf6)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:58.636605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc4c61-c5fc-4d9b-9871-b93d610f7013 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:58.641456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37151e-78b5-42c0-80c8-3aa3050bb6b4 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:58.667225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37151e-78b5-42c0-80c8-3aa3050bb6b4 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:58.677972 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2b873-e1eb-4b15-8a29-f259de62dcb0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6720518c03413aa8f85e4d6142fe87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:58.679309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2b873-e1eb-4b15-8a29-f259de62dcb0 service placement] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:58.679633 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2b873-e1eb-4b15-8a29-f259de62dcb0 service placement] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:58.680387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2b873-e1eb-4b15-8a29-f259de62dcb0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:58.699773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2b873-e1eb-4b15-8a29-f259de62dcb0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:58.702448 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2b873-e1eb-4b15-8a29-f259de62dcb0 service placement] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:58.739490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 813/1610] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:49:58.837741 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ecdc4c61-c5fc-4d9b-9871-b93d610f7013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0692c115a2264edda61b9373bf1b7cf6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:58.838884 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ecdc4c61-c5fc-4d9b-9871-b93d610f7013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0692c115a2264edda61b9373bf1b7cf6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:58.842074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ecdc4c61-c5fc-4d9b-9871-b93d610f7013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0692c115a2264edda61b9373bf1b7cf6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:58.897416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ecdc4c61-c5fc-4d9b-9871-b93d610f7013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0692c115a2264edda61b9373bf1b7cf6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:58.911493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:58.922536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ecdc4c61-c5fc-4d9b-9871-b93d610f7013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0692c115a2264edda61b9373bf1b7cf6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:58.980878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 798/1611] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:49:58 2022] DELETE /identity/v3/users/0692c115a2264edda61b9373bf1b7cf6 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:58.987175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48abe98-f402-4356-914b-01df9d9f6754 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:58.987656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48abe98-f402-4356-914b-01df9d9f6754 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:58.988163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48abe98-f402-4356-914b-01df9d9f6754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:59.050245 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-e48abe98-f402-4356-914b-01df9d9f6754 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:49:59.051853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e48abe98-f402-4356-914b-01df9d9f6754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:59.202985 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc4c61-c5fc-4d9b-9871-b93d610f7013 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:59.206220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ecdc4c61-c5fc-4d9b-9871-b93d610f7013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:59.251980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 814/1612] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:49:59.271888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48abe98-f402-4356-914b-01df9d9f6754 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:59.271888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e48abe98-f402-4356-914b-01df9d9f6754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:49:59.382587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc4c61-c5fc-4d9b-9871-b93d610f7013 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:59.383914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48abe98-f402-4356-914b-01df9d9f6754 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:59.388269 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be684bc-0acb-4b4d-ba64-99fc210a7e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:59.389145 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be684bc-0acb-4b4d-ba64-99fc210a7e24 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:59.389479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be684bc-0acb-4b4d-ba64-99fc210a7e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:59.389938 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be684bc-0acb-4b4d-ba64-99fc210a7e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:59.393077 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:59.395127 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:59.395642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:59.396109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] PATH_INFO: `/v3/projects/9c0fca1ba6844ea0b6f3a3ea311b98b1` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:59.398022 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c0fca1ba6844ea0b6f3a3ea311b98b1)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:59.400149 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:59.403155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be684bc-0acb-4b4d-ba64-99fc210a7e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:59.405076 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be684bc-0acb-4b4d-ba64-99fc210a7e24 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:59.421878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 815/1613] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:49:59.469289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c0fca1ba6844ea0b6f3a3ea311b98b1', 'request_id': 'req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:59.482738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c0fca1ba6844ea0b6f3a3ea311b98b1', 'request_id': 'req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:59.498902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c0fca1ba6844ea0b6f3a3ea311b98b1', 'request_id': 'req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:59.516444 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] The token cache is being invalidate because project 9c0fca1ba6844ea0b6f3a3ea311b98b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:49:59.516697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:59.517544 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:59.519223 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 799/1614] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:49:59 2022] DELETE /identity/v3/projects/9c0fca1ba6844ea0b6f3a3ea311b98b1 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:49:59.523752 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be684bc-0acb-4b4d-ba64-99fc210a7e24 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:49:59.535174 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be684bc-0acb-4b4d-ba64-99fc210a7e24 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:49:59.594095 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:49:59.613975 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764e09e-ddcc-489f-bd11-fc920b3d14e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:49:59.614638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764e09e-ddcc-489f-bd11-fc920b3d14e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:49:59.614910 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764e09e-ddcc-489f-bd11-fc920b3d14e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:49:59.615185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764e09e-ddcc-489f-bd11-fc920b3d14e3 admin admin] PATH_INFO: `/v3/users/a202b55ac8244d8cac6cb3821673e7c0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:49:59.616479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f764e09e-ddcc-489f-bd11-fc920b3d14e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a202b55ac8244d8cac6cb3821673e7c0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:49:59.618482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f764e09e-ddcc-489f-bd11-fc920b3d14e3 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:49:59.689626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f764e09e-ddcc-489f-bd11-fc920b3d14e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a202b55ac8244d8cac6cb3821673e7c0'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:59.699032 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f764e09e-ddcc-489f-bd11-fc920b3d14e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a202b55ac8244d8cac6cb3821673e7c0'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:59.699825 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f764e09e-ddcc-489f-bd11-fc920b3d14e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a202b55ac8244d8cac6cb3821673e7c0'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:59.701465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f764e09e-ddcc-489f-bd11-fc920b3d14e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a202b55ac8244d8cac6cb3821673e7c0'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:59.714376 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f764e09e-ddcc-489f-bd11-fc920b3d14e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a202b55ac8244d8cac6cb3821673e7c0'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:49:59.751493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 816/1615] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:49:59 2022] DELETE /identity/v3/users/a202b55ac8244d8cac6cb3821673e7c0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:50:00.280128 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:00.282501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:00.336586 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9919f9-ab34-40e7-ac7f-489b3b9bdc50 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:00.378675 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764e09e-ddcc-489f-bd11-fc920b3d14e3 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:00.381357 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f764e09e-ddcc-489f-bd11-fc920b3d14e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:00.497608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:00.498493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:00.499226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:00.499599 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] PATH_INFO: `/v3/projects/dcea1503d9354b29b8cb7af672c41470` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:00.502350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcea1503d9354b29b8cb7af672c41470)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:00.504893 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:00.540530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:00.575266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764e09e-ddcc-489f-bd11-fc920b3d14e3 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:00.609695 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcea1503d9354b29b8cb7af672c41470', 'request_id': 'req-e7b83bc8-8986-4b59-8f26-fe6a61915e49'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.625765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcea1503d9354b29b8cb7af672c41470', 'request_id': 'req-e7b83bc8-8986-4b59-8f26-fe6a61915e49'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.647491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcea1503d9354b29b8cb7af672c41470', 'request_id': 'req-e7b83bc8-8986-4b59-8f26-fe6a61915e49'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.679745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] The token cache is being invalidate because project dcea1503d9354b29b8cb7af672c41470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:00.680041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.681296 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.684492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 800/1616] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:00 2022] DELETE /identity/v3/projects/dcea1503d9354b29b8cb7af672c41470 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:00.685857 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:00.688378 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:00.710645 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66545a91-d6a4-4f0f-84fd-15dcde444cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:00.711935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66545a91-d6a4-4f0f-84fd-15dcde444cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:00.712600 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66545a91-d6a4-4f0f-84fd-15dcde444cac admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:00.713318 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66545a91-d6a4-4f0f-84fd-15dcde444cac admin admin] PATH_INFO: `/v3/users/10364e42b429400fb014eb5f6d60e3bb` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:00.717862 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66545a91-d6a4-4f0f-84fd-15dcde444cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=10364e42b429400fb014eb5f6d60e3bb)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:00.722467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66545a91-d6a4-4f0f-84fd-15dcde444cac admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:00.741953 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:00.798151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-66545a91-d6a4-4f0f-84fd-15dcde444cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10364e42b429400fb014eb5f6d60e3bb'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.808707 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b83bc8-8986-4b59-8f26-fe6a61915e49 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:00.810933 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-66545a91-d6a4-4f0f-84fd-15dcde444cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10364e42b429400fb014eb5f6d60e3bb'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.812051 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-66545a91-d6a4-4f0f-84fd-15dcde444cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10364e42b429400fb014eb5f6d60e3bb'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.813052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-66545a91-d6a4-4f0f-84fd-15dcde444cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10364e42b429400fb014eb5f6d60e3bb'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.816675 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:00.817688 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:00.818037 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:00.818518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] PATH_INFO: `/v3/projects/3587b1ffcb3f4067804b1eb4d873daf3` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:00.820820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3587b1ffcb3f4067804b1eb4d873daf3)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:00.823459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:00.827596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-66545a91-d6a4-4f0f-84fd-15dcde444cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10364e42b429400fb014eb5f6d60e3bb'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.851842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 817/1617] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:00 2022] DELETE /identity/v3/users/10364e42b429400fb014eb5f6d60e3bb => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:00.857113 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66545a91-d6a4-4f0f-84fd-15dcde444cac admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:00.858479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-66545a91-d6a4-4f0f-84fd-15dcde444cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:00.913055 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3587b1ffcb3f4067804b1eb4d873daf3', 'request_id': 'req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.921863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3587b1ffcb3f4067804b1eb4d873daf3', 'request_id': 'req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.933690 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3587b1ffcb3f4067804b1eb4d873daf3', 'request_id': 'req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.946996 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] The token cache is being invalidate because project 3587b1ffcb3f4067804b1eb4d873daf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:00.947393 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.948271 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:00.950089 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 801/1618] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:00 2022] DELETE /identity/v3/projects/3587b1ffcb3f4067804b1eb4d873daf3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:00.953914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:00.957135 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:00.959968 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66545a91-d6a4-4f0f-84fd-15dcde444cac admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:01.040416 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ec38c-4310-45a4-b2bf-9eb3e7592501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:01.041484 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ec38c-4310-45a4-b2bf-9eb3e7592501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:01.041988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ec38c-4310-45a4-b2bf-9eb3e7592501 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:01.042327 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ec38c-4310-45a4-b2bf-9eb3e7592501 admin admin] PATH_INFO: `/v3/users/c065b783c56843e7b9648f859bce2ec1` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:01.043817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ec38c-4310-45a4-b2bf-9eb3e7592501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c065b783c56843e7b9648f859bce2ec1)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:01.046010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ec38c-4310-45a4-b2bf-9eb3e7592501 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:01.058750 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:01.097616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:01.105804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42d7be-ba2f-465e-8af8-1a5614a1fd74 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:01.109297 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-090ec38c-4310-45a4-b2bf-9eb3e7592501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c065b783c56843e7b9648f859bce2ec1'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.114134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b6de8-f069-4e52-a96c-eb7168c679c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:01.114978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b6de8-f069-4e52-a96c-eb7168c679c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:01.115298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b6de8-f069-4e52-a96c-eb7168c679c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:01.115704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b6de8-f069-4e52-a96c-eb7168c679c5 admin admin] PATH_INFO: `/v3/users/d68ae56243644473a9aef337c5a3eca7` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:01.117386 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b6de8-f069-4e52-a96c-eb7168c679c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d68ae56243644473a9aef337c5a3eca7)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:01.119063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-090ec38c-4310-45a4-b2bf-9eb3e7592501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c065b783c56843e7b9648f859bce2ec1'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.119397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b6de8-f069-4e52-a96c-eb7168c679c5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:01.121049 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-090ec38c-4310-45a4-b2bf-9eb3e7592501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c065b783c56843e7b9648f859bce2ec1'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.121860 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-090ec38c-4310-45a4-b2bf-9eb3e7592501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c065b783c56843e7b9648f859bce2ec1'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.135479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-090ec38c-4310-45a4-b2bf-9eb3e7592501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c065b783c56843e7b9648f859bce2ec1'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.171786 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 818/1619] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:00 2022] DELETE /identity/v3/users/c065b783c56843e7b9648f859bce2ec1 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:01.210758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-105b6de8-f069-4e52-a96c-eb7168c679c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd68ae56243644473a9aef337c5a3eca7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.212029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-105b6de8-f069-4e52-a96c-eb7168c679c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd68ae56243644473a9aef337c5a3eca7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.213299 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-105b6de8-f069-4e52-a96c-eb7168c679c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd68ae56243644473a9aef337c5a3eca7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.240407 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-105b6de8-f069-4e52-a96c-eb7168c679c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd68ae56243644473a9aef337c5a3eca7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.251998 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-105b6de8-f069-4e52-a96c-eb7168c679c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd68ae56243644473a9aef337c5a3eca7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.264599 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ec38c-4310-45a4-b2bf-9eb3e7592501 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:01.267243 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-090ec38c-4310-45a4-b2bf-9eb3e7592501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:01.281644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 802/1620] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:00 2022] DELETE /identity/v3/users/d68ae56243644473a9aef337c5a3eca7 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:50:01.407806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ec38c-4310-45a4-b2bf-9eb3e7592501 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:01.533014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:01.534097 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:01.534656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:01.535037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] PATH_INFO: `/v3/projects/3d590755103141aaa003054cb889a183` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:01.537097 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d590755103141aaa003054cb889a183)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:01.540808 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:01.612173 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d590755103141aaa003054cb889a183', 'request_id': 'req-7b1229f3-de87-46ef-a209-8630de036695'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.624398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d590755103141aaa003054cb889a183', 'request_id': 'req-7b1229f3-de87-46ef-a209-8630de036695'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.637767 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d590755103141aaa003054cb889a183', 'request_id': 'req-7b1229f3-de87-46ef-a209-8630de036695'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.641338 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:01.683373 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] The token cache is being invalidate because project 3d590755103141aaa003054cb889a183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:01.684925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.686985 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:01.696012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 819/1621] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:01 2022] DELETE /identity/v3/projects/3d590755103141aaa003054cb889a183 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:50:01.696931 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b6de8-f069-4e52-a96c-eb7168c679c5 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:01.699784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-105b6de8-f069-4e52-a96c-eb7168c679c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:01.750110 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b6de8-f069-4e52-a96c-eb7168c679c5 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:01.766460 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:01.769492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:01.907766 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefd636-20e5-4c7d-be67-72314c5a520f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:01.908519 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefd636-20e5-4c7d-be67-72314c5a520f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:01.908829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefd636-20e5-4c7d-be67-72314c5a520f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:01.909092 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefd636-20e5-4c7d-be67-72314c5a520f admin admin] PATH_INFO: `/v3/users/b9335df2fedc4f09a654fe8ed5b1c3e9` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:01.912510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefd636-20e5-4c7d-be67-72314c5a520f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9335df2fedc4f09a654fe8ed5b1c3e9)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:01.917280 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefd636-20e5-4c7d-be67-72314c5a520f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:01.968585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1229f3-de87-46ef-a209-8630de036695 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:01.977361 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:01.978963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:01.979399 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:01.980047 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa admin admin] PATH_INFO: `/v3/projects/a1e5848dda374b96a551bd65c5df01c5` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:01.982773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1e5848dda374b96a551bd65c5df01c5)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:01.985108 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:02.009604 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bdefd636-20e5-4c7d-be67-72314c5a520f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9335df2fedc4f09a654fe8ed5b1c3e9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.010521 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bdefd636-20e5-4c7d-be67-72314c5a520f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9335df2fedc4f09a654fe8ed5b1c3e9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.011189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bdefd636-20e5-4c7d-be67-72314c5a520f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9335df2fedc4f09a654fe8ed5b1c3e9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.026726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:02.039158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bdefd636-20e5-4c7d-be67-72314c5a520f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9335df2fedc4f09a654fe8ed5b1c3e9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.052331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bdefd636-20e5-4c7d-be67-72314c5a520f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9335df2fedc4f09a654fe8ed5b1c3e9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.054203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1e5848dda374b96a551bd65c5df01c5', 'request_id': 'req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.066201 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1e5848dda374b96a551bd65c5df01c5', 'request_id': 'req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.075565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 803/1622] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:01 2022] DELETE /identity/v3/users/b9335df2fedc4f09a654fe8ed5b1c3e9 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:02.075565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefd636-20e5-4c7d-be67-72314c5a520f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:02.075565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bdefd636-20e5-4c7d-be67-72314c5a520f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:02.077883 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1e5848dda374b96a551bd65c5df01c5', 'request_id': 'req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.098197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa admin admin] The token cache is being invalidate because project a1e5848dda374b96a551bd65c5df01c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:02.098658 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.101341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ef37568f-e831-41b5-b7ee-d3f5ebcb2dfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.104837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 820/1623] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:01 2022] DELETE /identity/v3/projects/a1e5848dda374b96a551bd65c5df01c5 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:50:02.113859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b86b71-8eab-4cb2-9245-1763aad0485f None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.114620 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b86b71-8eab-4cb2-9245-1763aad0485f None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.116168 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b86b71-8eab-4cb2-9245-1763aad0485f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.149409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-14b86b71-8eab-4cb2-9245-1763aad0485f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:02.151733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-14b86b71-8eab-4cb2-9245-1763aad0485f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:02.170964 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefd636-20e5-4c7d-be67-72314c5a520f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:02.176447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:02.177133 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.177413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.177759 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da admin admin] PATH_INFO: `/v3/projects/c1dbf9e4805b4a3ca46ce2440778203e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.179222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1dbf9e4805b4a3ca46ce2440778203e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:02.179377 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 821/1624] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:02.181485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:02.200844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ea84-2942-4557-aefd-e53376da49f8 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.201324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ea84-2942-4557-aefd-e53376da49f8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.201833 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ea84-2942-4557-aefd-e53376da49f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.248248 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-4126ea84-2942-4557-aefd-e53376da49f8 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:02.248962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1dbf9e4805b4a3ca46ce2440778203e', 'request_id': 'req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.250656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4126ea84-2942-4557-aefd-e53376da49f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:02.258233 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1dbf9e4805b4a3ca46ce2440778203e', 'request_id': 'req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.270787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1dbf9e4805b4a3ca46ce2440778203e', 'request_id': 'req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.287003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da admin admin] The token cache is being invalidate because project c1dbf9e4805b4a3ca46ce2440778203e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:02.288212 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.289307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d9bbf4b1-4d3c-4fac-99b5-b2c1098040da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.291646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 804/1625] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:02 2022] DELETE /identity/v3/projects/c1dbf9e4805b4a3ca46ce2440778203e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:02.300420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ea978-6031-4df0-9ca6-600bca8cfea5 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.301028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ea978-6031-4df0-9ca6-600bca8cfea5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.301466 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ea978-6031-4df0-9ca6-600bca8cfea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.327187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b34ea978-6031-4df0-9ca6-600bca8cfea5 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:02.328742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b34ea978-6031-4df0-9ca6-600bca8cfea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:02.370460 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 822/1626] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:50:02.375920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126ea84-2942-4557-aefd-e53376da49f8 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:02.379082 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4126ea84-2942-4557-aefd-e53376da49f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:02.393769 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126ea84-2942-4557-aefd-e53376da49f8 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:02.401689 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:02.402873 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.403045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.403242 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] PATH_INFO: `/v3/projects/a6ebf4f6aff24a52b2a576688f0ed547` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.404772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ebf4f6aff24a52b2a576688f0ed547)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:02.406852 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:02.413748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 805/1627] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:02.416629 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ea978-6031-4df0-9ca6-600bca8cfea5 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:02.419485 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:02.438942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ea978-6031-4df0-9ca6-600bca8cfea5 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:02.464788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce09818-faf9-43b1-87fb-97eede74d4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:02.465467 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce09818-faf9-43b1-87fb-97eede74d4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.465895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce09818-faf9-43b1-87fb-97eede74d4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.466331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce09818-faf9-43b1-87fb-97eede74d4e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.467036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce09818-faf9-43b1-87fb-97eede74d4e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:50:02.467316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce09818-faf9-43b1-87fb-97eede74d4e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:02.472200 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce09818-faf9-43b1-87fb-97eede74d4e0 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:02.484714 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:02.491423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 806/1628] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:50:02.491423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253b667-10f9-44e5-86b3-e336b3cf7bec None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.491423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253b667-10f9-44e5-86b3-e336b3cf7bec None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.491423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253b667-10f9-44e5-86b3-e336b3cf7bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.500202 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ebf4f6aff24a52b2a576688f0ed547', 'request_id': 'req-af63ef9c-2800-40c1-85c6-e324073c598c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.513453 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ebf4f6aff24a52b2a576688f0ed547', 'request_id': 'req-af63ef9c-2800-40c1-85c6-e324073c598c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.521517 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-3253b667-10f9-44e5-86b3-e336b3cf7bec None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:02.523070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3253b667-10f9-44e5-86b3-e336b3cf7bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:02.526976 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ebf4f6aff24a52b2a576688f0ed547', 'request_id': 'req-af63ef9c-2800-40c1-85c6-e324073c598c'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.543114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] The token cache is being invalidate because project a6ebf4f6aff24a52b2a576688f0ed547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:02.543560 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.544781 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:02.556011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 823/1629] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:02 2022] DELETE /identity/v3/projects/a6ebf4f6aff24a52b2a576688f0ed547 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:02.556011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:02.556011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:02.639819 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 807/1630] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:02.641811 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa30dd1-5f43-4d0f-8809-aed76a8f1314 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.642301 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa30dd1-5f43-4d0f-8809-aed76a8f1314 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.642568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa30dd1-5f43-4d0f-8809-aed76a8f1314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.656024 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63ef9c-2800-40c1-85c6-e324073c598c admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:02.665524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-aaa30dd1-5f43-4d0f-8809-aed76a8f1314 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:02.666765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-aaa30dd1-5f43-4d0f-8809-aed76a8f1314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:02.667070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf26e6e-6969-4972-b17b-24903f18ee81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:02.668295 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf26e6e-6969-4972-b17b-24903f18ee81 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.668846 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf26e6e-6969-4972-b17b-24903f18ee81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.669399 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf26e6e-6969-4972-b17b-24903f18ee81 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.670578 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf26e6e-6969-4972-b17b-24903f18ee81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:02.675232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 808/1631] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:02.676189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf26e6e-6969-4972-b17b-24903f18ee81 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:02.681189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa30dd1-5f43-4d0f-8809-aed76a8f1314 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:02.694107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa30dd1-5f43-4d0f-8809-aed76a8f1314 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:02.700899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03d862-08aa-49c6-a38a-4a0b1aec2a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:02.701580 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03d862-08aa-49c6-a38a-4a0b1aec2a04 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.701879 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03d862-08aa-49c6-a38a-4a0b1aec2a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.702114 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03d862-08aa-49c6-a38a-4a0b1aec2a04 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.702778 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03d862-08aa-49c6-a38a-4a0b1aec2a04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:50:02.703073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03d862-08aa-49c6-a38a-4a0b1aec2a04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:02.705023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03d862-08aa-49c6-a38a-4a0b1aec2a04 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:02.720558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 809/1632] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:50:02.721692 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a26812-319d-4aff-ac52-1ffca449b0c4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.721977 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a26812-319d-4aff-ac52-1ffca449b0c4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.722245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a26812-319d-4aff-ac52-1ffca449b0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.724887 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 824/1633] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:02 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:02.725728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf26e6e-6969-4972-b17b-24903f18ee81 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:02.746545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-31a26812-319d-4aff-ac52-1ffca449b0c4 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:02.747812 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-31a26812-319d-4aff-ac52-1ffca449b0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:02.825713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 810/1634] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:02.826668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a26812-319d-4aff-ac52-1ffca449b0c4 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:02.837750 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a26812-319d-4aff-ac52-1ffca449b0c4 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:02.843328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c902cbf-e368-4d52-a517-b6bac283950f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:02.844007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c902cbf-e368-4d52-a517-b6bac283950f admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.844308 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c902cbf-e368-4d52-a517-b6bac283950f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.844588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c902cbf-e368-4d52-a517-b6bac283950f admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.845233 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c902cbf-e368-4d52-a517-b6bac283950f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:02.847217 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c902cbf-e368-4d52-a517-b6bac283950f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:02.848726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf26e6e-6969-4972-b17b-24903f18ee81 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:02.856563 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02179b-dcec-424f-ba10-ad321e932dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:02.857213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02179b-dcec-424f-ba10-ad321e932dca admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.857490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02179b-dcec-424f-ba10-ad321e932dca admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.857761 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02179b-dcec-424f-ba10-ad321e932dca admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.858405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02179b-dcec-424f-ba10-ad321e932dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:02.860409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02179b-dcec-424f-ba10-ad321e932dca admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:02.876837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 811/1635] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:02 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:02.876837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c902cbf-e368-4d52-a517-b6bac283950f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:02.888483 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c902cbf-e368-4d52-a517-b6bac283950f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:02.896859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206adde-a0ac-4cc7-bcea-68f8f4575e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:02.897804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206adde-a0ac-4cc7-bcea-68f8f4575e6a admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.898284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206adde-a0ac-4cc7-bcea-68f8f4575e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.898670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206adde-a0ac-4cc7-bcea-68f8f4575e6a admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.898817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 825/1636] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:02 2022] POST /identity/v3/projects => generated 401 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:02.899356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206adde-a0ac-4cc7-bcea-68f8f4575e6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:50:02.899556 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02179b-dcec-424f-ba10-ad321e932dca admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:02.899679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206adde-a0ac-4cc7-bcea-68f8f4575e6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:02.902165 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206adde-a0ac-4cc7-bcea-68f8f4575e6a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:02.910979 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:02.922953 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 812/1637] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:50:02.922953 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206adde-a0ac-4cc7-bcea-68f8f4575e6a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:02.934817 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206adde-a0ac-4cc7-bcea-68f8f4575e6a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:02.980207 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02179b-dcec-424f-ba10-ad321e932dca admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:02.983467 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:02.985640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05137ca-5d63-49f6-a98d-5b288c849440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:02.986870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05137ca-5d63-49f6-a98d-5b288c849440 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:02.986870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05137ca-5d63-49f6-a98d-5b288c849440 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:02.986870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05137ca-5d63-49f6-a98d-5b288c849440 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:02.987926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05137ca-5d63-49f6-a98d-5b288c849440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:02.990069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05137ca-5d63-49f6-a98d-5b288c849440 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:03.007735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 826/1638] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:03.007735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05137ca-5d63-49f6-a98d-5b288c849440 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:03.008418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a6060-d9fd-4aba-a499-bbb6aa71ffda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:03.008418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a6060-d9fd-4aba-a499-bbb6aa71ffda admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.008418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a6060-d9fd-4aba-a499-bbb6aa71ffda admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.008418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a6060-d9fd-4aba-a499-bbb6aa71ffda admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.009038 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a6060-d9fd-4aba-a499-bbb6aa71ffda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:03.011625 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a6060-d9fd-4aba-a499-bbb6aa71ffda admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:03.015066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05137ca-5d63-49f6-a98d-5b288c849440 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:03.020378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2edae2-fda5-4149-8820-2f66bb4e49e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:03.021172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2edae2-fda5-4149-8820-2f66bb4e49e0 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.021552 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2edae2-fda5-4149-8820-2f66bb4e49e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.021951 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2edae2-fda5-4149-8820-2f66bb4e49e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.022985 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2edae2-fda5-4149-8820-2f66bb4e49e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:03.025474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2edae2-fda5-4149-8820-2f66bb4e49e0 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:03.044341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 813/1639] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:02 2022] POST /identity/v3/users => generated 301 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:03.046994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a6060-d9fd-4aba-a499-bbb6aa71ffda admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:03.067098 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 827/1640] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:03 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:03.067098 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2edae2-fda5-4149-8820-2f66bb4e49e0 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:03.067448 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a6060-d9fd-4aba-a499-bbb6aa71ffda admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:03.161250 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdfc17-a5a3-460a-b3f8-7eac9029ef84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:03.162190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdfc17-a5a3-460a-b3f8-7eac9029ef84 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.162605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdfc17-a5a3-460a-b3f8-7eac9029ef84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.162989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdfc17-a5a3-460a-b3f8-7eac9029ef84 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.163781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdfc17-a5a3-460a-b3f8-7eac9029ef84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:03.165581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdfc17-a5a3-460a-b3f8-7eac9029ef84 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:03.167664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2edae2-fda5-4149-8820-2f66bb4e49e0 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:03.176665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2e213-969c-4f3f-955e-e101a1ab169a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:03.177710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2e213-969c-4f3f-955e-e101a1ab169a admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.178194 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2e213-969c-4f3f-955e-e101a1ab169a admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.178799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 814/1641] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:03.180835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdfc17-a5a3-460a-b3f8-7eac9029ef84 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:03.181076 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2e213-969c-4f3f-955e-e101a1ab169a admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.181076 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2e213-969c-4f3f-955e-e101a1ab169a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:03.183069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf2e213-969c-4f3f-955e-e101a1ab169a admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:03.194866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdfc17-a5a3-460a-b3f8-7eac9029ef84 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:03.195982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 828/1642] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:03.196878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2e213-969c-4f3f-955e-e101a1ab169a admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:03.207452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a9c75-ac7a-4cb3-9010-ece577e3d0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:03.208073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a9c75-ac7a-4cb3-9010-ece577e3d0db admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.208437 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a9c75-ac7a-4cb3-9010-ece577e3d0db admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.208647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a9c75-ac7a-4cb3-9010-ece577e3d0db admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.209271 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a9c75-ac7a-4cb3-9010-ece577e3d0db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:03.209427 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2e213-969c-4f3f-955e-e101a1ab169a admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:03.211588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a9c75-ac7a-4cb3-9010-ece577e3d0db admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:03.215549 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198edba5-1ea4-40ff-b3d0-941a0c33f8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:03.216694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198edba5-1ea4-40ff-b3d0-941a0c33f8ef admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.217020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198edba5-1ea4-40ff-b3d0-941a0c33f8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.217358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198edba5-1ea4-40ff-b3d0-941a0c33f8ef admin admin] PATH_INFO: `/v3/projects/b58d317ad84e46be85ed15b3e51c2f97/users/af1daddbc1b84ab79f269e124cbea4fb/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.241157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 815/1643] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:03 2022] POST /identity/v3/users => generated 293 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:03.242125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a9c75-ac7a-4cb3-9010-ece577e3d0db admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:03.242801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198edba5-1ea4-40ff-b3d0-941a0c33f8ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58d317ad84e46be85ed15b3e51c2f97, user_id=af1daddbc1b84ab79f269e124cbea4fb, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:03.245801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198edba5-1ea4-40ff-b3d0-941a0c33f8ef admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:03.254525 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a9c75-ac7a-4cb3-9010-ece577e3d0db admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:03.260301 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79577a-caf1-4c39-85ee-fb310aeeb068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:03.260948 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79577a-caf1-4c39-85ee-fb310aeeb068 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.261228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79577a-caf1-4c39-85ee-fb310aeeb068 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.261503 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79577a-caf1-4c39-85ee-fb310aeeb068 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.262177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79577a-caf1-4c39-85ee-fb310aeeb068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:03.263933 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79577a-caf1-4c39-85ee-fb310aeeb068 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:03.267220 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 829/1644] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:03 2022] PUT /identity/v3/projects/b58d317ad84e46be85ed15b3e51c2f97/users/af1daddbc1b84ab79f269e124cbea4fb/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:03.272019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198edba5-1ea4-40ff-b3d0-941a0c33f8ef admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:03.281179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 816/1645] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:03.285991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e8025-f01e-4a54-9690-7a2af089a111 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.286270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e8025-f01e-4a54-9690-7a2af089a111 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.286530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e8025-f01e-4a54-9690-7a2af089a111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.308406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-117e8025-f01e-4a54-9690-7a2af089a111 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:03.309838 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-117e8025-f01e-4a54-9690-7a2af089a111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:03.352427 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:03.360632 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198edba5-1ea4-40ff-b3d0-941a0c33f8ef admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:03.366738 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a3839-af78-4583-9c8b-af3a72403a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:03.367480 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a3839-af78-4583-9c8b-af3a72403a78 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.367899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a3839-af78-4583-9c8b-af3a72403a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.368310 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a3839-af78-4583-9c8b-af3a72403a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.369315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a3839-af78-4583-9c8b-af3a72403a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:03.371384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a3839-af78-4583-9c8b-af3a72403a78 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:03.386654 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 830/1646] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:03.387847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 817/1647] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:03.388852 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a3839-af78-4583-9c8b-af3a72403a78 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:03.392539 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e8025-f01e-4a54-9690-7a2af089a111 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:03.404548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e8025-f01e-4a54-9690-7a2af089a111 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:03.404794 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a3839-af78-4583-9c8b-af3a72403a78 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:03.411257 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20db43-dd1d-4450-92a7-be29aff870e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:03.411613 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb15401-2719-4795-bd7c-d15a8a71f6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:03.412425 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20db43-dd1d-4450-92a7-be29aff870e3 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.412783 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb15401-2719-4795-bd7c-d15a8a71f6c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.412876 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20db43-dd1d-4450-92a7-be29aff870e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.413240 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20db43-dd1d-4450-92a7-be29aff870e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.413339 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb15401-2719-4795-bd7c-d15a8a71f6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.413660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb15401-2719-4795-bd7c-d15a8a71f6c7 admin admin] PATH_INFO: `/v3/projects/f493a21b298f44deaf88816ba4a8da75/users/b1e7195be11b4240bbbe5d7c77e0a4ba/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.414280 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20db43-dd1d-4450-92a7-be29aff870e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:03.417258 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20db43-dd1d-4450-92a7-be29aff870e3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:03.432149 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 818/1648] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:03.433090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20db43-dd1d-4450-92a7-be29aff870e3 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:03.437953 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8e20db43-dd1d-4450-92a7-be29aff870e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:03.441922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb15401-2719-4795-bd7c-d15a8a71f6c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f493a21b298f44deaf88816ba4a8da75, user_id=b1e7195be11b4240bbbe5d7c77e0a4ba, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:03.447787 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb15401-2719-4795-bd7c-d15a8a71f6c7 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:03.469440 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 831/1649] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:03 2022] PUT /identity/v3/projects/f493a21b298f44deaf88816ba4a8da75/users/b1e7195be11b4240bbbe5d7c77e0a4ba/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:03.470374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb15401-2719-4795-bd7c-d15a8a71f6c7 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:03.486604 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:03.529305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20db43-dd1d-4450-92a7-be29aff870e3 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:03.564362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb15401-2719-4795-bd7c-d15a8a71f6c7 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:03.570223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22be503-441d-40ef-9019-fe043b04ca33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:03.571280 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22be503-441d-40ef-9019-fe043b04ca33 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.571730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22be503-441d-40ef-9019-fe043b04ca33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.572210 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22be503-441d-40ef-9019-fe043b04ca33 admin admin] PATH_INFO: `/v3/projects/bca8becce97b4fdb8c5dd1c753693b96/users/04a644c95fbc4b62b8cdd027f1edb6d4/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.597262 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22be503-441d-40ef-9019-fe043b04ca33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca8becce97b4fdb8c5dd1c753693b96, user_id=04a644c95fbc4b62b8cdd027f1edb6d4, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:03.602214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22be503-441d-40ef-9019-fe043b04ca33 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:03.622240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 832/1650] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:03 2022] PUT /identity/v3/projects/bca8becce97b4fdb8c5dd1c753693b96/users/04a644c95fbc4b62b8cdd027f1edb6d4/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:03.625850 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5537b-19ec-4c06-8fb2-27d35ebf1842 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.626298 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5537b-19ec-4c06-8fb2-27d35ebf1842 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.626836 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5537b-19ec-4c06-8fb2-27d35ebf1842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.667966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-3da5537b-19ec-4c06-8fb2-27d35ebf1842 None None] MFA Rules not processed for user `b1e7195be11b4240bbbe5d7c77e0a4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:03.669420 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3da5537b-19ec-4c06-8fb2-27d35ebf1842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:03.746140 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408b714-2d5e-4f75-b8a0-9c3df2035db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:03.746787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408b714-2d5e-4f75-b8a0-9c3df2035db7 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.747088 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408b714-2d5e-4f75-b8a0-9c3df2035db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.747391 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408b714-2d5e-4f75-b8a0-9c3df2035db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.750872 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 833/1651] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:03 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:03.753491 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427fcd4-2569-4737-a9dc-02673c9e315c None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.754133 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427fcd4-2569-4737-a9dc-02673c9e315c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.754664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427fcd4-2569-4737-a9dc-02673c9e315c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.760558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408b714-2d5e-4f75-b8a0-9c3df2035db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:03.762282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408b714-2d5e-4f75-b8a0-9c3df2035db7 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:03.789313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 819/1652] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:03.791163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408b714-2d5e-4f75-b8a0-9c3df2035db7 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:03.799522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-8427fcd4-2569-4737-a9dc-02673c9e315c None None] MFA Rules not processed for user `04a644c95fbc4b62b8cdd027f1edb6d4`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:03.801002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8427fcd4-2569-4737-a9dc-02673c9e315c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:03.820505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408b714-2d5e-4f75-b8a0-9c3df2035db7 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:03.847089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:03.939404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 834/1653] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:03.941316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7588ae-ea26-4007-87fd-c3d97df568e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:03.942023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7588ae-ea26-4007-87fd-c3d97df568e7 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:03.942412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7588ae-ea26-4007-87fd-c3d97df568e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:03.942676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7588ae-ea26-4007-87fd-c3d97df568e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:03.943700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7588ae-ea26-4007-87fd-c3d97df568e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:03.945870 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7588ae-ea26-4007-87fd-c3d97df568e7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:03.952048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427fcd4-2569-4737-a9dc-02673c9e315c None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:03.957497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8427fcd4-2569-4737-a9dc-02673c9e315c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:04.011660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 820/1654] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:03 2022] POST /identity/v3/projects => generated 399 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:04.018785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7588ae-ea26-4007-87fd-c3d97df568e7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:04.055808 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427fcd4-2569-4737-a9dc-02673c9e315c None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:04.127092 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1a50d-b472-4524-baac-daa530adc9ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:04.127736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1a50d-b472-4524-baac-daa530adc9ae service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:04.128074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1a50d-b472-4524-baac-daa530adc9ae service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:04.128349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1a50d-b472-4524-baac-daa530adc9ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:04.150098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7588ae-ea26-4007-87fd-c3d97df568e7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:04.156657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dabc3d-2c16-4e04-acd4-da11f08f0de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:04.157408 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dabc3d-2c16-4e04-acd4-da11f08f0de3 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:04.157757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dabc3d-2c16-4e04-acd4-da11f08f0de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:04.158104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dabc3d-2c16-4e04-acd4-da11f08f0de3 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:04.158848 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dabc3d-2c16-4e04-acd4-da11f08f0de3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:04.161030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dabc3d-2c16-4e04-acd4-da11f08f0de3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:04.196924 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 821/1655] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:04 2022] POST /identity/v3/users => generated 306 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:50:04.198700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dabc3d-2c16-4e04-acd4-da11f08f0de3 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:04.207250 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1a50d-b472-4524-baac-daa530adc9ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:04.209028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1a50d-b472-4524-baac-daa530adc9ae service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:04.211613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:04.222337 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dabc3d-2c16-4e04-acd4-da11f08f0de3 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:04.226832 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 835/1656] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:04.238071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc958c0-6b62-44bb-be19-9e60a8c2efa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:04.238800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc958c0-6b62-44bb-be19-9e60a8c2efa9 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:04.239511 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc958c0-6b62-44bb-be19-9e60a8c2efa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:04.239511 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc958c0-6b62-44bb-be19-9e60a8c2efa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:04.240070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc958c0-6b62-44bb-be19-9e60a8c2efa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:04.241641 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc958c0-6b62-44bb-be19-9e60a8c2efa9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:04.251151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1a50d-b472-4524-baac-daa530adc9ae service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:04.264656 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 822/1657] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:50:04.270830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc958c0-6b62-44bb-be19-9e60a8c2efa9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:04.272349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1a50d-b472-4524-baac-daa530adc9ae service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:04.277966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dfac5-eb63-431d-a80a-031424f5a19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:04.279353 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dfac5-eb63-431d-a80a-031424f5a19d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:04.279956 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dfac5-eb63-431d-a80a-031424f5a19d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:04.280378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dfac5-eb63-431d-a80a-031424f5a19d admin admin] PATH_INFO: `/v3/users/04a644c95fbc4b62b8cdd027f1edb6d4` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:04.281836 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dfac5-eb63-431d-a80a-031424f5a19d admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a644c95fbc4b62b8cdd027f1edb6d4)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:04.284055 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dfac5-eb63-431d-a80a-031424f5a19d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:04.299922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc958c0-6b62-44bb-be19-9e60a8c2efa9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:04.322377 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d22873-c2b1-4a42-95b9-2534b932fb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:04.327047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d22873-c2b1-4a42-95b9-2534b932fb89 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:04.327047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d22873-c2b1-4a42-95b9-2534b932fb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:04.327792 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d22873-c2b1-4a42-95b9-2534b932fb89 admin admin] PATH_INFO: `/v3/projects/be9c3b63e7084ebea439b7814d0d6c1e/users/5ae8a1daefe34f9a80ad502b3dea3441/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:04.374452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d22873-c2b1-4a42-95b9-2534b932fb89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be9c3b63e7084ebea439b7814d0d6c1e, user_id=5ae8a1daefe34f9a80ad502b3dea3441, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:04.387568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d22873-c2b1-4a42-95b9-2534b932fb89 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:04.406054 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:04.420860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 823/1658] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:04 2022] PUT /identity/v3/projects/be9c3b63e7084ebea439b7814d0d6c1e/users/5ae8a1daefe34f9a80ad502b3dea3441/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:04.426830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-cb0dfac5-eb63-431d-a80a-031424f5a19d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a644c95fbc4b62b8cdd027f1edb6d4'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:04.427106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d22873-c2b1-4a42-95b9-2534b932fb89 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:04.438239 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-cb0dfac5-eb63-431d-a80a-031424f5a19d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a644c95fbc4b62b8cdd027f1edb6d4'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:04.439270 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-cb0dfac5-eb63-431d-a80a-031424f5a19d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a644c95fbc4b62b8cdd027f1edb6d4'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:04.442130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-cb0dfac5-eb63-431d-a80a-031424f5a19d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a644c95fbc4b62b8cdd027f1edb6d4'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:04.456083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-cb0dfac5-eb63-431d-a80a-031424f5a19d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a644c95fbc4b62b8cdd027f1edb6d4'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:04.501823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 836/1659] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:04 2022] DELETE /identity/v3/users/04a644c95fbc4b62b8cdd027f1edb6d4 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:50:04.508063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36127f95-236a-4b07-80ad-f8a34b5fa214 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:04.508449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36127f95-236a-4b07-80ad-f8a34b5fa214 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:04.509150 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36127f95-236a-4b07-80ad-f8a34b5fa214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:04.540192 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-36127f95-236a-4b07-80ad-f8a34b5fa214 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:04.541518 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-36127f95-236a-4b07-80ad-f8a34b5fa214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:04.563143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d22873-c2b1-4a42-95b9-2534b932fb89 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:04.629031 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 837/1660] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:04.640170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36127f95-236a-4b07-80ad-f8a34b5fa214 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:04.642212 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-36127f95-236a-4b07-80ad-f8a34b5fa214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:04.703084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dd8fd-c527-4b97-af91-a0c15dacfde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:04.704027 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dd8fd-c527-4b97-af91-a0c15dacfde4 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:04.704697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dd8fd-c527-4b97-af91-a0c15dacfde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:04.704937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dd8fd-c527-4b97-af91-a0c15dacfde4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:04.706006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68dd8fd-c527-4b97-af91-a0c15dacfde4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:04.708126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68dd8fd-c527-4b97-af91-a0c15dacfde4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:04.724386 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 824/1661] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:04.730991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dd8fd-c527-4b97-af91-a0c15dacfde4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:04.733674 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f68dd8fd-c527-4b97-af91-a0c15dacfde4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:04.766079 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dd8fd-c527-4b97-af91-a0c15dacfde4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:04.786923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf91dd-c3dc-47c1-a156-a71e6f32baec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:04.789249 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf91dd-c3dc-47c1-a156-a71e6f32baec admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:04.789249 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf91dd-c3dc-47c1-a156-a71e6f32baec admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:04.789603 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf91dd-c3dc-47c1-a156-a71e6f32baec admin admin] PATH_INFO: `/v3/projects/be9c3b63e7084ebea439b7814d0d6c1e/users/5ae8a1daefe34f9a80ad502b3dea3441/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:04.791922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf91dd-c3dc-47c1-a156-a71e6f32baec admin admin] RBAC: Authorizing `identity:create_grant(project_id=be9c3b63e7084ebea439b7814d0d6c1e, user_id=5ae8a1daefe34f9a80ad502b3dea3441, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:04.795275 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf91dd-c3dc-47c1-a156-a71e6f32baec admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:04.804444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36127f95-236a-4b07-80ad-f8a34b5fa214 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:04.810441 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701b886-250f-40cf-8923-35435519bee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:04.811431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701b886-250f-40cf-8923-35435519bee7 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:04.812746 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701b886-250f-40cf-8923-35435519bee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:04.813162 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701b886-250f-40cf-8923-35435519bee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:04.819078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 825/1662] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:04 2022] PUT /identity/v3/projects/be9c3b63e7084ebea439b7814d0d6c1e/users/5ae8a1daefe34f9a80ad502b3dea3441/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:50:04.834296 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67486a4-74c5-40b4-996d-6eb13a36f1fc None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:04.834773 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67486a4-74c5-40b4-996d-6eb13a36f1fc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:04.834959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701b886-250f-40cf-8923-35435519bee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:04.835114 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67486a4-74c5-40b4-996d-6eb13a36f1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:04.836575 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701b886-250f-40cf-8923-35435519bee7 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:04.851188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 838/1663] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:04.894698 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e67486a4-74c5-40b4-996d-6eb13a36f1fc None None] MFA Rules not processed for user `5ae8a1daefe34f9a80ad502b3dea3441`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:04.896757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e67486a4-74c5-40b4-996d-6eb13a36f1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:04.984729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:05.023086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 826/1664] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:04 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:05.036654 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701b886-250f-40cf-8923-35435519bee7 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:05.038705 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3701b886-250f-40cf-8923-35435519bee7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:05.145518 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701b886-250f-40cf-8923-35435519bee7 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:05.171146 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:05.238320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eab23c-4d71-4867-977c-79181ff10996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:05.239013 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eab23c-4d71-4867-977c-79181ff10996 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:05.239299 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eab23c-4d71-4867-977c-79181ff10996 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:05.239601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eab23c-4d71-4867-977c-79181ff10996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:05.263747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eab23c-4d71-4867-977c-79181ff10996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:05.271787 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eab23c-4d71-4867-977c-79181ff10996 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:05.296051 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 839/1665] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:05 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:05.313207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021a57a-ed5f-4c85-95fb-1652f0786223 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:05.313700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021a57a-ed5f-4c85-95fb-1652f0786223 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:05.314359 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021a57a-ed5f-4c85-95fb-1652f0786223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:05.427237 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2021a57a-ed5f-4c85-95fb-1652f0786223 None None] MFA Rules not processed for user `af1daddbc1b84ab79f269e124cbea4fb`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:05.428835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2021a57a-ed5f-4c85-95fb-1652f0786223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:05.433554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eab23c-4d71-4867-977c-79181ff10996 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:05.435409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-81eab23c-4d71-4867-977c-79181ff10996 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:05.539391 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 827/1666] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:05.545331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021a57a-ed5f-4c85-95fb-1652f0786223 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:05.547176 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2021a57a-ed5f-4c85-95fb-1652f0786223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:05.561659 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021a57a-ed5f-4c85-95fb-1652f0786223 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:05.566319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0486a16-b5bd-4fca-b527-dbf4ce853172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:05.567245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0486a16-b5bd-4fca-b527-dbf4ce853172 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:05.567519 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0486a16-b5bd-4fca-b527-dbf4ce853172 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:05.568304 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0486a16-b5bd-4fca-b527-dbf4ce853172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:05.570029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e0486a16-b5bd-4fca-b527-dbf4ce853172 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:05.604547 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eab23c-4d71-4867-977c-79181ff10996 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:05.612725 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2d094-e601-43f8-b4bd-413e03796b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:05.613821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2d094-e601-43f8-b4bd-413e03796b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:05.614743 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2d094-e601-43f8-b4bd-413e03796b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:05.615737 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2d094-e601-43f8-b4bd-413e03796b80 admin admin] PATH_INFO: `/v3/projects/bca8becce97b4fdb8c5dd1c753693b96` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:05.618017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be2d094-e601-43f8-b4bd-413e03796b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bca8becce97b4fdb8c5dd1c753693b96)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:05.622445 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be2d094-e601-43f8-b4bd-413e03796b80 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:05.643824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0486a16-b5bd-4fca-b527-dbf4ce853172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:05.645816 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0486a16-b5bd-4fca-b527-dbf4ce853172 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:05.661583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 828/1667] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:05 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:05.703435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4be2d094-e601-43f8-b4bd-413e03796b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bca8becce97b4fdb8c5dd1c753693b96', 'request_id': 'req-4be2d094-e601-43f8-b4bd-413e03796b80'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:05.715260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4be2d094-e601-43f8-b4bd-413e03796b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bca8becce97b4fdb8c5dd1c753693b96', 'request_id': 'req-4be2d094-e601-43f8-b4bd-413e03796b80'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:05.731254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4be2d094-e601-43f8-b4bd-413e03796b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bca8becce97b4fdb8c5dd1c753693b96', 'request_id': 'req-4be2d094-e601-43f8-b4bd-413e03796b80'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:05.755762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4be2d094-e601-43f8-b4bd-413e03796b80 admin admin] The token cache is being invalidate because project bca8becce97b4fdb8c5dd1c753693b96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:05.756190 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4be2d094-e601-43f8-b4bd-413e03796b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:05.756969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4be2d094-e601-43f8-b4bd-413e03796b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:05.758797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 840/1668] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:05 2022] DELETE /identity/v3/projects/bca8becce97b4fdb8c5dd1c753693b96 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:50:05.765000 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bff08f-7ca5-4279-88b8-3678dd1fcfd7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:05.765464 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bff08f-7ca5-4279-88b8-3678dd1fcfd7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:05.765942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bff08f-7ca5-4279-88b8-3678dd1fcfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:05.797426 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b2bff08f-7ca5-4279-88b8-3678dd1fcfd7 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:05.799181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b2bff08f-7ca5-4279-88b8-3678dd1fcfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:05.881852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 829/1669] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:05.900456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256015d2-8b02-4876-8a40-918fb1575012 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:05.900456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256015d2-8b02-4876-8a40-918fb1575012 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:05.900456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256015d2-8b02-4876-8a40-918fb1575012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:05.929505 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-256015d2-8b02-4876-8a40-918fb1575012 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:05.931395 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-256015d2-8b02-4876-8a40-918fb1575012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:05.942210 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 841/1670] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:05.948651 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bff08f-7ca5-4279-88b8-3678dd1fcfd7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:05.962758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bff08f-7ca5-4279-88b8-3678dd1fcfd7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:05.969604 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f954c-45d6-4fdc-a286-15ff3495d0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:05.970297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f954c-45d6-4fdc-a286-15ff3495d0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:05.970560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f954c-45d6-4fdc-a286-15ff3495d0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:05.970829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f954c-45d6-4fdc-a286-15ff3495d0f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:05.971486 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f954c-45d6-4fdc-a286-15ff3495d0f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:50:05.971774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f954c-45d6-4fdc-a286-15ff3495d0f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:05.974465 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f954c-45d6-4fdc-a286-15ff3495d0f1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:05.992288 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256015d2-8b02-4876-8a40-918fb1575012 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:05.993983 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-256015d2-8b02-4876-8a40-918fb1575012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:06.004602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:06.018006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 830/1671] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:50:06.019788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f954c-45d6-4fdc-a286-15ff3495d0f1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:06.038693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f954c-45d6-4fdc-a286-15ff3495d0f1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:06.046521 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07de1d3-c1b1-42e8-a13c-2df51a924448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:06.047605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07de1d3-c1b1-42e8-a13c-2df51a924448 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:06.048325 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07de1d3-c1b1-42e8-a13c-2df51a924448 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:06.048804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07de1d3-c1b1-42e8-a13c-2df51a924448 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:06.049983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07de1d3-c1b1-42e8-a13c-2df51a924448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:06.054285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07de1d3-c1b1-42e8-a13c-2df51a924448 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:06.099648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:06.125096 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 831/1672] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:06 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:06.129400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07de1d3-c1b1-42e8-a13c-2df51a924448 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:06.139293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256015d2-8b02-4876-8a40-918fb1575012 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:06.213684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f76e96-d4bb-4467-b069-3397ad41d19a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:06.214332 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f76e96-d4bb-4467-b069-3397ad41d19a service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:06.214613 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f76e96-d4bb-4467-b069-3397ad41d19a service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:06.214883 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f76e96-d4bb-4467-b069-3397ad41d19a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:06.216950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c9f76e96-d4bb-4467-b069-3397ad41d19a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:06.219256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07de1d3-c1b1-42e8-a13c-2df51a924448 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:06.225238 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d350033-1786-4bcc-9435-02885d758683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:06.226086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d350033-1786-4bcc-9435-02885d758683 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:06.226422 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d350033-1786-4bcc-9435-02885d758683 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:06.226703 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d350033-1786-4bcc-9435-02885d758683 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:06.227357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d350033-1786-4bcc-9435-02885d758683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:06.229872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d350033-1786-4bcc-9435-02885d758683 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:06.263133 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 832/1673] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:06 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:06.269641 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d350033-1786-4bcc-9435-02885d758683 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:06.289139 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d350033-1786-4bcc-9435-02885d758683 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:06.297715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193fd54-ccc8-44b1-839f-fed6ce575dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:06.298933 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193fd54-ccc8-44b1-839f-fed6ce575dbd admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:06.299425 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193fd54-ccc8-44b1-839f-fed6ce575dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:06.300044 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193fd54-ccc8-44b1-839f-fed6ce575dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:06.301143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193fd54-ccc8-44b1-839f-fed6ce575dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:06.304440 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193fd54-ccc8-44b1-839f-fed6ce575dbd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:06.319198 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 833/1674] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:06.323649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193fd54-ccc8-44b1-839f-fed6ce575dbd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:06.336839 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193fd54-ccc8-44b1-839f-fed6ce575dbd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:06.340282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f76e96-d4bb-4467-b069-3397ad41d19a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:06.342682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f76e96-d4bb-4467-b069-3397ad41d19a service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:06.349746 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeedac4-b022-42b4-93aa-bd71a9bd1999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:06.349746 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeedac4-b022-42b4-93aa-bd71a9bd1999 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:06.349746 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeedac4-b022-42b4-93aa-bd71a9bd1999 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:06.349746 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeedac4-b022-42b4-93aa-bd71a9bd1999 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:06.350093 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeedac4-b022-42b4-93aa-bd71a9bd1999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:06.350093 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeedac4-b022-42b4-93aa-bd71a9bd1999 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:06.362216 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 842/1675] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:50:06.363897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 834/1676] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:50:06.370115 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f76e96-d4bb-4467-b069-3397ad41d19a service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:06.386659 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f76e96-d4bb-4467-b069-3397ad41d19a service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:06.393417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571cfcb-d5ee-4812-9cb5-66259a1116d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:06.394308 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571cfcb-d5ee-4812-9cb5-66259a1116d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:06.394708 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571cfcb-d5ee-4812-9cb5-66259a1116d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:06.394866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571cfcb-d5ee-4812-9cb5-66259a1116d9 admin admin] PATH_INFO: `/v3/projects/8c6271208c83488a8c9d2c1f483dcc24/users/ee70afee57f845828b840870752a579f/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:06.418903 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571cfcb-d5ee-4812-9cb5-66259a1116d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c6271208c83488a8c9d2c1f483dcc24, user_id=ee70afee57f845828b840870752a579f, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:06.422525 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571cfcb-d5ee-4812-9cb5-66259a1116d9 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:06.461607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd8b50-24f6-4711-9018-4b69f00407e0 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:06.461607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd8b50-24f6-4711-9018-4b69f00407e0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:06.461607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd8b50-24f6-4711-9018-4b69f00407e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:06.462878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 843/1677] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:06 2022] PUT /identity/v3/projects/8c6271208c83488a8c9d2c1f483dcc24/users/ee70afee57f845828b840870752a579f/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:06.492488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-fefd8b50-24f6-4711-9018-4b69f00407e0 None None] MFA Rules not processed for user `ee70afee57f845828b840870752a579f`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:06.493947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fefd8b50-24f6-4711-9018-4b69f00407e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:06.620513 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 835/1678] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:06.642589 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571cfcb-d5ee-4812-9cb5-66259a1116d9 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:06.645504 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b571cfcb-d5ee-4812-9cb5-66259a1116d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:06.754318 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571cfcb-d5ee-4812-9cb5-66259a1116d9 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:06.759102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba7c75-67dc-4292-b5c9-fde11593c1e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:06.759821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba7c75-67dc-4292-b5c9-fde11593c1e5 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:06.760118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba7c75-67dc-4292-b5c9-fde11593c1e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:06.760386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba7c75-67dc-4292-b5c9-fde11593c1e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:06.772191 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba7c75-67dc-4292-b5c9-fde11593c1e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:06.774595 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba7c75-67dc-4292-b5c9-fde11593c1e5 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:06.788388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 844/1679] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:50:07.967575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989d89-2b40-42ea-adf6-09b5ed6e7906 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:07.968181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989d89-2b40-42ea-adf6-09b5ed6e7906 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:07.968181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989d89-2b40-42ea-adf6-09b5ed6e7906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:07.996545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7b989d89-2b40-42ea-adf6-09b5ed6e7906 None None] MFA Rules not processed for user `af1daddbc1b84ab79f269e124cbea4fb`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:07.998169 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7b989d89-2b40-42ea-adf6-09b5ed6e7906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:08.041224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:08.071281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 836/1680] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:50:16.780812 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=87,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:18.063386 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=9 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:22.798575 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba7c75-67dc-4292-b5c9-fde11593c1e5 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:22.800194 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0bba7c75-67dc-4292-b5c9-fde11593c1e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:22.888227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba7c75-67dc-4292-b5c9-fde11593c1e5 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:22.894182 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c09d6-d5f4-4adc-8d9f-c86ef351cc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:22.894842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c09d6-d5f4-4adc-8d9f-c86ef351cc37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:22.895133 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c09d6-d5f4-4adc-8d9f-c86ef351cc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:22.895395 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c09d6-d5f4-4adc-8d9f-c86ef351cc37 admin admin] PATH_INFO: `/v3/users/b1e7195be11b4240bbbe5d7c77e0a4ba` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:22.896565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c09d6-d5f4-4adc-8d9f-c86ef351cc37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1e7195be11b4240bbbe5d7c77e0a4ba)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:22.899313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c09d6-d5f4-4adc-8d9f-c86ef351cc37 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:22.991423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c05c09d6-d5f4-4adc-8d9f-c86ef351cc37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1e7195be11b4240bbbe5d7c77e0a4ba'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:23.006726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c05c09d6-d5f4-4adc-8d9f-c86ef351cc37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1e7195be11b4240bbbe5d7c77e0a4ba'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:23.007907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c05c09d6-d5f4-4adc-8d9f-c86ef351cc37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1e7195be11b4240bbbe5d7c77e0a4ba'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:23.008957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c05c09d6-d5f4-4adc-8d9f-c86ef351cc37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1e7195be11b4240bbbe5d7c77e0a4ba'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:23.026447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c05c09d6-d5f4-4adc-8d9f-c86ef351cc37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1e7195be11b4240bbbe5d7c77e0a4ba'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:23.053713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 845/1681] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:22 2022] DELETE /identity/v3/users/b1e7195be11b4240bbbe5d7c77e0a4ba => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:23.058609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3a0eb-8ba6-42dc-bf6a-e3eb308dd76b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:23.060166 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3a0eb-8ba6-42dc-bf6a-e3eb308dd76b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:23.060575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3a0eb-8ba6-42dc-bf6a-e3eb308dd76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:23.086208 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e5d3a0eb-8ba6-42dc-bf6a-e3eb308dd76b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:23.087784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e5d3a0eb-8ba6-42dc-bf6a-e3eb308dd76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:23.175762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 837/1682] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:50:23.194219 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c09d6-d5f4-4adc-8d9f-c86ef351cc37 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:23.196813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c05c09d6-d5f4-4adc-8d9f-c86ef351cc37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:23.286430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c09d6-d5f4-4adc-8d9f-c86ef351cc37 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:23.291648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e4d74-f09f-42ef-9033-b81c4d13df2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:23.292316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e4d74-f09f-42ef-9033-b81c4d13df2f service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:23.292597 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e4d74-f09f-42ef-9033-b81c4d13df2f service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:23.292889 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e4d74-f09f-42ef-9033-b81c4d13df2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:23.309184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e4d74-f09f-42ef-9033-b81c4d13df2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:23.311244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e4d74-f09f-42ef-9033-b81c4d13df2f service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:23.325473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 846/1683] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:24.099301 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3a0eb-8ba6-42dc-bf6a-e3eb308dd76b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:24.101043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e5d3a0eb-8ba6-42dc-bf6a-e3eb308dd76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:24.115352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3a0eb-8ba6-42dc-bf6a-e3eb308dd76b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:24.127541 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:24.130867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:24.131550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:24.132097 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] PATH_INFO: `/v3/projects/f493a21b298f44deaf88816ba4a8da75` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:24.134479 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f493a21b298f44deaf88816ba4a8da75)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:24.138989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:24.204446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f493a21b298f44deaf88816ba4a8da75', 'request_id': 'req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:24.212370 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f493a21b298f44deaf88816ba4a8da75', 'request_id': 'req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:24.222632 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f493a21b298f44deaf88816ba4a8da75', 'request_id': 'req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:24.238451 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] The token cache is being invalidate because project f493a21b298f44deaf88816ba4a8da75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:24.238750 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:24.239623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:24.241267 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 838/1684] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:24 2022] DELETE /identity/v3/projects/f493a21b298f44deaf88816ba4a8da75 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:24.246465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e4d74-f09f-42ef-9033-b81c4d13df2f service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:24.260410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e4d74-f09f-42ef-9033-b81c4d13df2f service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:24.284365 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:24.340021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b23c1-0d05-405f-b8f6-2b942897b6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:24.340591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b23c1-0d05-405f-b8f6-2b942897b6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:24.341081 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b23c1-0d05-405f-b8f6-2b942897b6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:24.341360 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b23c1-0d05-405f-b8f6-2b942897b6f7 admin admin] PATH_INFO: `/v3/users/5ae8a1daefe34f9a80ad502b3dea3441` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:24.342879 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b23c1-0d05-405f-b8f6-2b942897b6f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ae8a1daefe34f9a80ad502b3dea3441)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:24.344957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b23c1-0d05-405f-b8f6-2b942897b6f7 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:24.413407 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-229b23c1-0d05-405f-b8f6-2b942897b6f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ae8a1daefe34f9a80ad502b3dea3441'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:24.425110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-229b23c1-0d05-405f-b8f6-2b942897b6f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ae8a1daefe34f9a80ad502b3dea3441'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:24.426033 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-229b23c1-0d05-405f-b8f6-2b942897b6f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ae8a1daefe34f9a80ad502b3dea3441'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:24.426909 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-229b23c1-0d05-405f-b8f6-2b942897b6f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ae8a1daefe34f9a80ad502b3dea3441'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:24.446926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-229b23c1-0d05-405f-b8f6-2b942897b6f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ae8a1daefe34f9a80ad502b3dea3441'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:24.485671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 847/1685] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:24 2022] DELETE /identity/v3/users/5ae8a1daefe34f9a80ad502b3dea3441 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:24.775906 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:24.778222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:24.841783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba58299-4e6c-4265-a2e8-40c4a167a7d2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:25.012236 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b23c1-0d05-405f-b8f6-2b942897b6f7 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:25.013959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-229b23c1-0d05-405f-b8f6-2b942897b6f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:25.124155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808754b-6a48-4ee7-bd6c-d222dd115945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:25.125071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808754b-6a48-4ee7-bd6c-d222dd115945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:25.125071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808754b-6a48-4ee7-bd6c-d222dd115945 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:25.125356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808754b-6a48-4ee7-bd6c-d222dd115945 admin admin] PATH_INFO: `/v3/users/ee70afee57f845828b840870752a579f` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:25.126608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808754b-6a48-4ee7-bd6c-d222dd115945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee70afee57f845828b840870752a579f)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:25.128698 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808754b-6a48-4ee7-bd6c-d222dd115945 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:25.142582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b23c1-0d05-405f-b8f6-2b942897b6f7 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:25.154736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208de51-2e68-443a-9bee-154d28926230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:25.159168 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208de51-2e68-443a-9bee-154d28926230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:25.160328 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208de51-2e68-443a-9bee-154d28926230 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:25.162879 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208de51-2e68-443a-9bee-154d28926230 admin admin] PATH_INFO: `/v3/users/451ad89c9fa241ef86818b7bbe0f26de` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:25.172841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208de51-2e68-443a-9bee-154d28926230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=451ad89c9fa241ef86818b7bbe0f26de)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:25.178184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208de51-2e68-443a-9bee-154d28926230 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:25.199681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:25.238200 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4808754b-6a48-4ee7-bd6c-d222dd115945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee70afee57f845828b840870752a579f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:25.238863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4808754b-6a48-4ee7-bd6c-d222dd115945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee70afee57f845828b840870752a579f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:25.239575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4808754b-6a48-4ee7-bd6c-d222dd115945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee70afee57f845828b840870752a579f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:25.265349 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4808754b-6a48-4ee7-bd6c-d222dd115945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee70afee57f845828b840870752a579f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:25.282330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4808754b-6a48-4ee7-bd6c-d222dd115945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee70afee57f845828b840870752a579f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:25.326064 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 839/1686] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:24 2022] DELETE /identity/v3/users/ee70afee57f845828b840870752a579f => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:25.331934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92ad3e-6f4a-4004-9796-bfb915fd1bc9 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:25.332224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92ad3e-6f4a-4004-9796-bfb915fd1bc9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:25.332736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92ad3e-6f4a-4004-9796-bfb915fd1bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:25.393663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ae92ad3e-6f4a-4004-9796-bfb915fd1bc9 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:25.398621 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ae92ad3e-6f4a-4004-9796-bfb915fd1bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:25.412203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:25.421232 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7208de51-2e68-443a-9bee-154d28926230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '451ad89c9fa241ef86818b7bbe0f26de'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:25.452883 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7208de51-2e68-443a-9bee-154d28926230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '451ad89c9fa241ef86818b7bbe0f26de'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:25.456795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7208de51-2e68-443a-9bee-154d28926230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '451ad89c9fa241ef86818b7bbe0f26de'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:25.460070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7208de51-2e68-443a-9bee-154d28926230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '451ad89c9fa241ef86818b7bbe0f26de'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:25.480517 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7208de51-2e68-443a-9bee-154d28926230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '451ad89c9fa241ef86818b7bbe0f26de'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:25.528382 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 848/1687] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:25 2022] DELETE /identity/v3/users/451ad89c9fa241ef86818b7bbe0f26de => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:25.529946 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208de51-2e68-443a-9bee-154d28926230 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:25.531648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7208de51-2e68-443a-9bee-154d28926230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:25.577388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 840/1688] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:50:25.578903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0d568-4cce-40a2-b44d-9682545116cf None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:25.581647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0d568-4cce-40a2-b44d-9682545116cf None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:25.581903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0d568-4cce-40a2-b44d-9682545116cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:25.642822 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0ad0d568-4cce-40a2-b44d-9682545116cf None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:25.644078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0ad0d568-4cce-40a2-b44d-9682545116cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:25.808416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 841/1689] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:25.809934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0d568-4cce-40a2-b44d-9682545116cf None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:25.812740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0ad0d568-4cce-40a2-b44d-9682545116cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:25.877932 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208de51-2e68-443a-9bee-154d28926230 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:25.890605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:25.891757 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:25.892214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:25.892621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] PATH_INFO: `/v3/projects/be9c3b63e7084ebea439b7814d0d6c1e` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:25.894470 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be9c3b63e7084ebea439b7814d0d6c1e)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:25.897026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:25.963254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be9c3b63e7084ebea439b7814d0d6c1e', 'request_id': 'req-9854e87c-2d7a-4eb5-9351-46974042c943'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:25.983743 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be9c3b63e7084ebea439b7814d0d6c1e', 'request_id': 'req-9854e87c-2d7a-4eb5-9351-46974042c943'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:25.987820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:25.996632 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be9c3b63e7084ebea439b7814d0d6c1e', 'request_id': 'req-9854e87c-2d7a-4eb5-9351-46974042c943'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:26.007929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0d568-4cce-40a2-b44d-9682545116cf None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:26.023999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] The token cache is being invalidate because project be9c3b63e7084ebea439b7814d0d6c1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:26.023999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:26.023999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:26.023999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 849/1690] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:25 2022] DELETE /identity/v3/projects/be9c3b63e7084ebea439b7814d0d6c1e => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:26.023999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:26.024480 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:26.112146 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21033461-97ff-49fd-a5ad-799c29ccece7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:26.113248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21033461-97ff-49fd-a5ad-799c29ccece7 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:26.113740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21033461-97ff-49fd-a5ad-799c29ccece7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:26.114260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21033461-97ff-49fd-a5ad-799c29ccece7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:26.117962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-21033461-97ff-49fd-a5ad-799c29ccece7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:26.125229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854e87c-2d7a-4eb5-9351-46974042c943 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:26.131110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbc51f-813a-4d99-a5c2-7dfa13d55c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:26.132238 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbc51f-813a-4d99-a5c2-7dfa13d55c76 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:26.133814 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbc51f-813a-4d99-a5c2-7dfa13d55c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:26.134747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbc51f-813a-4d99-a5c2-7dfa13d55c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:26.137301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-5bcbc51f-813a-4d99-a5c2-7dfa13d55c76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:26.143934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:26.245809 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21033461-97ff-49fd-a5ad-799c29ccece7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:26.247424 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21033461-97ff-49fd-a5ad-799c29ccece7 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:26.252180 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbc51f-813a-4d99-a5c2-7dfa13d55c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:26.254468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbc51f-813a-4d99-a5c2-7dfa13d55c76 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:26.266302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 842/1691] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:26.269137 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96a71b-1807-4641-bf25-36259f847e10 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:26.269435 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96a71b-1807-4641-bf25-36259f847e10 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:26.269707 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96a71b-1807-4641-bf25-36259f847e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:26.271664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 850/1692] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:26.296873 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-fb96a71b-1807-4641-bf25-36259f847e10 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:26.298294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fb96a71b-1807-4641-bf25-36259f847e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:26.308617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 843/1693] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:26.317862 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7e2ee-ff7a-40d4-8412-add27f6acfea None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:26.318180 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7e2ee-ff7a-40d4-8412-add27f6acfea None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:26.318444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7e2ee-ff7a-40d4-8412-add27f6acfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:26.358352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-43a7e2ee-ff7a-40d4-8412-add27f6acfea None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:26.361320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-43a7e2ee-ff7a-40d4-8412-add27f6acfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:26.385944 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 851/1694] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:26.387777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96a71b-1807-4641-bf25-36259f847e10 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:26.404497 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96a71b-1807-4641-bf25-36259f847e10 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:26.418990 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cbf47-3928-452a-b87f-eca92c3b0954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:26.420365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cbf47-3928-452a-b87f-eca92c3b0954 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:26.421194 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cbf47-3928-452a-b87f-eca92c3b0954 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:26.423283 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cbf47-3928-452a-b87f-eca92c3b0954 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:26.425574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cbf47-3928-452a-b87f-eca92c3b0954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:50:26.427062 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cbf47-3928-452a-b87f-eca92c3b0954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:26.435143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cbf47-3928-452a-b87f-eca92c3b0954 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:26.469468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 844/1695] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:50:26.471092 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7e2ee-ff7a-40d4-8412-add27f6acfea None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:26.486662 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7e2ee-ff7a-40d4-8412-add27f6acfea None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:26.495566 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bedd0a-663d-4a6b-838a-0b5a49646da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:26.500050 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bedd0a-663d-4a6b-838a-0b5a49646da6 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:26.500746 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bedd0a-663d-4a6b-838a-0b5a49646da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:26.501029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bedd0a-663d-4a6b-838a-0b5a49646da6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:26.501710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bedd0a-663d-4a6b-838a-0b5a49646da6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:26.503990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bedd0a-663d-4a6b-838a-0b5a49646da6 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:26.555939 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 852/1696] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:26 2022] POST /identity/v3/projects => generated 385 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:26.562498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cbf47-3928-452a-b87f-eca92c3b0954 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:26.577699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cbf47-3928-452a-b87f-eca92c3b0954 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:26.692837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3f568-2f3e-4643-9a8f-803cb09ef4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:26.693477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3f568-2f3e-4643-9a8f-803cb09ef4ad admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:26.693750 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3f568-2f3e-4643-9a8f-803cb09ef4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:26.694021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3f568-2f3e-4643-9a8f-803cb09ef4ad admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:26.694665 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3f568-2f3e-4643-9a8f-803cb09ef4ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:26.696730 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3f568-2f3e-4643-9a8f-803cb09ef4ad admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:26.740944 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 845/1697] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:26 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:26.744339 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bedd0a-663d-4a6b-838a-0b5a49646da6 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:26.766422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bedd0a-663d-4a6b-838a-0b5a49646da6 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:26.778497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797903e-51b7-424a-8623-76ad25316f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:26.780225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797903e-51b7-424a-8623-76ad25316f35 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:26.781152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797903e-51b7-424a-8623-76ad25316f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:26.781771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797903e-51b7-424a-8623-76ad25316f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:26.783417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797903e-51b7-424a-8623-76ad25316f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:26.785664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797903e-51b7-424a-8623-76ad25316f35 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:26.806378 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3f568-2f3e-4643-9a8f-803cb09ef4ad admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:26.817028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 853/1698] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:26.833068 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3f568-2f3e-4643-9a8f-803cb09ef4ad admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:26.843542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50991fa0-7ac5-4245-bb65-e3b06cc03485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:26.844895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50991fa0-7ac5-4245-bb65-e3b06cc03485 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:26.845423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50991fa0-7ac5-4245-bb65-e3b06cc03485 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:26.846097 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50991fa0-7ac5-4245-bb65-e3b06cc03485 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:26.847704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50991fa0-7ac5-4245-bb65-e3b06cc03485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:26.851418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50991fa0-7ac5-4245-bb65-e3b06cc03485 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:26.877749 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797903e-51b7-424a-8623-76ad25316f35 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:26.881116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8797903e-51b7-424a-8623-76ad25316f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:26.893300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 846/1699] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:26.899527 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50991fa0-7ac5-4245-bb65-e3b06cc03485 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:26.911541 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50991fa0-7ac5-4245-bb65-e3b06cc03485 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:26.917611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73c8ef-a811-4356-bbf8-765a7c578fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:26.920147 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797903e-51b7-424a-8623-76ad25316f35 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:26.922488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73c8ef-a811-4356-bbf8-765a7c578fee admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:26.922803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73c8ef-a811-4356-bbf8-765a7c578fee admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:26.923156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73c8ef-a811-4356-bbf8-765a7c578fee admin admin] PATH_INFO: `/v3/projects/15c81d35884a48a0925d7e8ce7d6df22/users/2c041502ff4f43eda21d905cc11e1229/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:26.929695 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:26.933506 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:26.934608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:26.936784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:26.937828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a admin admin] PATH_INFO: `/v3/projects/8c6271208c83488a8c9d2c1f483dcc24` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:26.942467 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c6271208c83488a8c9d2c1f483dcc24)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:26.956122 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:26.988485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73c8ef-a811-4356-bbf8-765a7c578fee admin admin] RBAC: Authorizing `identity:create_grant(project_id=15c81d35884a48a0925d7e8ce7d6df22, user_id=2c041502ff4f43eda21d905cc11e1229, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:26.995387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73c8ef-a811-4356-bbf8-765a7c578fee admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:27.023909 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 847/1700] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:26 2022] PUT /identity/v3/projects/15c81d35884a48a0925d7e8ce7d6df22/users/2c041502ff4f43eda21d905cc11e1229/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:27.038854 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c6271208c83488a8c9d2c1f483dcc24', 'request_id': 'req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:27.042653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c3578-5dbc-4591-ac8f-9807a3bec130 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:27.042947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c3578-5dbc-4591-ac8f-9807a3bec130 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:27.043224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c3578-5dbc-4591-ac8f-9807a3bec130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:27.054232 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c6271208c83488a8c9d2c1f483dcc24', 'request_id': 'req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:27.073329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c6271208c83488a8c9d2c1f483dcc24', 'request_id': 'req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:27.103221 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a admin admin] The token cache is being invalidate because project 8c6271208c83488a8c9d2c1f483dcc24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:27.103704 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:27.104814 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-b63e5355-d0c7-42dd-84ea-2f4bf6ed973a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:27.107244 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-4c9c3578-5dbc-4591-ac8f-9807a3bec130 None None] MFA Rules not processed for user `2c041502ff4f43eda21d905cc11e1229`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:27.107935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 854/1701] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:26 2022] DELETE /identity/v3/projects/8c6271208c83488a8c9d2c1f483dcc24 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:50:27.114782 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8041-53cc-4bf4-8a50-cf9177bf3e13 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:27.115091 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4c9c3578-5dbc-4591-ac8f-9807a3bec130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:27.116758 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8041-53cc-4bf4-8a50-cf9177bf3e13 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:27.117392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8041-53cc-4bf4-8a50-cf9177bf3e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:27.149755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-ff3d8041-53cc-4bf4-8a50-cf9177bf3e13 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:27.151147 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ff3d8041-53cc-4bf4-8a50-cf9177bf3e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:27.214401 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:27.233606 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 848/1702] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:27.240361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c3578-5dbc-4591-ac8f-9807a3bec130 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:27.243802 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4c9c3578-5dbc-4591-ac8f-9807a3bec130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:27.253922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 855/1703] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:27.261919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd182e54-1a09-448d-9a24-ed599cb2e2ae None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:27.262276 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd182e54-1a09-448d-9a24-ed599cb2e2ae None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:27.262525 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd182e54-1a09-448d-9a24-ed599cb2e2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:27.270178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c3578-5dbc-4591-ac8f-9807a3bec130 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:27.279312 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf7c5b-e6b6-4e14-855d-d4952629b8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:27.280418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf7c5b-e6b6-4e14-855d-d4952629b8ac admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:27.280818 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf7c5b-e6b6-4e14-855d-d4952629b8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:27.281292 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf7c5b-e6b6-4e14-855d-d4952629b8ac admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:27.282459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf7c5b-e6b6-4e14-855d-d4952629b8ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:27.286156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf7c5b-e6b6-4e14-855d-d4952629b8ac admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:27.287503 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-fd182e54-1a09-448d-9a24-ed599cb2e2ae None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:27.289555 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-fd182e54-1a09-448d-9a24-ed599cb2e2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:27.304225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 856/1704] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:50:27.305689 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd182e54-1a09-448d-9a24-ed599cb2e2ae None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:27.317870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd182e54-1a09-448d-9a24-ed599cb2e2ae None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:27.327525 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade165ef-683c-438a-b5b6-2764e9d37bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:27.329046 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade165ef-683c-438a-b5b6-2764e9d37bbd admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:27.330395 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade165ef-683c-438a-b5b6-2764e9d37bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:27.331141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade165ef-683c-438a-b5b6-2764e9d37bbd admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:27.332608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade165ef-683c-438a-b5b6-2764e9d37bbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:50:27.333141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade165ef-683c-438a-b5b6-2764e9d37bbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:27.336596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade165ef-683c-438a-b5b6-2764e9d37bbd admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:27.337794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 849/1705] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:27 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:50:27.344426 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf7c5b-e6b6-4e14-855d-d4952629b8ac admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:27.364291 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 857/1706] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:50:27.368528 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade165ef-683c-438a-b5b6-2764e9d37bbd admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:27.387508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade165ef-683c-438a-b5b6-2764e9d37bbd admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:27.460072 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf7c5b-e6b6-4e14-855d-d4952629b8ac admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:27.467985 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa05c8e-efb0-4c2f-8920-cced59a7f517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:27.468887 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa05c8e-efb0-4c2f-8920-cced59a7f517 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:27.469156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa05c8e-efb0-4c2f-8920-cced59a7f517 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:27.469425 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa05c8e-efb0-4c2f-8920-cced59a7f517 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:27.470067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa05c8e-efb0-4c2f-8920-cced59a7f517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:27.472761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa05c8e-efb0-4c2f-8920-cced59a7f517 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:27.502861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 850/1707] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:27 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:27.507087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d188e1d-99d9-4717-adc0-f35d500fdd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:27.507920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d188e1d-99d9-4717-adc0-f35d500fdd50 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:27.508203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d188e1d-99d9-4717-adc0-f35d500fdd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:27.508403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d188e1d-99d9-4717-adc0-f35d500fdd50 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:27.509304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d188e1d-99d9-4717-adc0-f35d500fdd50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:27.509880 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa05c8e-efb0-4c2f-8920-cced59a7f517 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:27.512044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d188e1d-99d9-4717-adc0-f35d500fdd50 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:27.526380 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa05c8e-efb0-4c2f-8920-cced59a7f517 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:27.540549 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6fb4d-8fb7-49d7-b06d-e65369b1ff35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:27.541305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6fb4d-8fb7-49d7-b06d-e65369b1ff35 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:27.541632 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6fb4d-8fb7-49d7-b06d-e65369b1ff35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:27.545015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6fb4d-8fb7-49d7-b06d-e65369b1ff35 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:27.545015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6fb4d-8fb7-49d7-b06d-e65369b1ff35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:27.545015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6fb4d-8fb7-49d7-b06d-e65369b1ff35 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:27.560809 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 851/1708] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:27.567630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6fb4d-8fb7-49d7-b06d-e65369b1ff35 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:27.579798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 858/1709] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:27 2022] POST /identity/v3/projects => generated 369 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:50:27.586022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d188e1d-99d9-4717-adc0-f35d500fdd50 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:27.589056 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6fb4d-8fb7-49d7-b06d-e65369b1ff35 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:27.641128 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:27.744102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d188e1d-99d9-4717-adc0-f35d500fdd50 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:27.750867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee0dd0-b0fe-44e4-a0ce-b0408e01af0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:27.752380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828d569-89c9-4ded-a921-239d2c526e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:27.753499 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828d569-89c9-4ded-a921-239d2c526e4c admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:27.753799 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828d569-89c9-4ded-a921-239d2c526e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:27.754318 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828d569-89c9-4ded-a921-239d2c526e4c admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:27.754814 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee0dd0-b0fe-44e4-a0ce-b0408e01af0b admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:27.755817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828d569-89c9-4ded-a921-239d2c526e4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:27.756228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee0dd0-b0fe-44e4-a0ce-b0408e01af0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:27.757668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828d569-89c9-4ded-a921-239d2c526e4c admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:27.758238 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee0dd0-b0fe-44e4-a0ce-b0408e01af0b admin admin] PATH_INFO: `/v3/projects/5f613cafcf984d5f8ca91ca26ba389f1/users/b976e32deafb422a83708c45f581e163/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:27.787566 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 859/1710] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:27 2022] POST /identity/v3/users => generated 285 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:50:27.789029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828d569-89c9-4ded-a921-239d2c526e4c admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:27.792524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee0dd0-b0fe-44e4-a0ce-b0408e01af0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f613cafcf984d5f8ca91ca26ba389f1, user_id=b976e32deafb422a83708c45f581e163, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:27.793212 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0828d569-89c9-4ded-a921-239d2c526e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:27.798527 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee0dd0-b0fe-44e4-a0ce-b0408e01af0b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:27.812339 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828d569-89c9-4ded-a921-239d2c526e4c admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:27.823095 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:27.825031 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:27.825818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:27.826458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] PATH_INFO: `/v3/projects/7b3fde4da4504db1b9927e26b8eadb60` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:27.827612 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 852/1711] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:27 2022] PUT /identity/v3/projects/5f613cafcf984d5f8ca91ca26ba389f1/users/b976e32deafb422a83708c45f581e163/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:27.831020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee0dd0-b0fe-44e4-a0ce-b0408e01af0b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:27.832251 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b3fde4da4504db1b9927e26b8eadb60)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:27.835404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:27.905348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:27.930968 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee0dd0-b0fe-44e4-a0ce-b0408e01af0b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:27.935373 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b3fde4da4504db1b9927e26b8eadb60', 'request_id': 'req-f1194ada-8d1f-40b8-89e2-ed64191bf267'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:27.947864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b3fde4da4504db1b9927e26b8eadb60', 'request_id': 'req-f1194ada-8d1f-40b8-89e2-ed64191bf267'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:27.957604 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b3fde4da4504db1b9927e26b8eadb60', 'request_id': 'req-f1194ada-8d1f-40b8-89e2-ed64191bf267'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:27.973415 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] The token cache is being invalidate because project 7b3fde4da4504db1b9927e26b8eadb60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:27.974246 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:27.975320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:27.977598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 860/1712] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:27 2022] DELETE /identity/v3/projects/7b3fde4da4504db1b9927e26b8eadb60 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:27.979597 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:27.982734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:28.053707 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9066866-caf7-4500-9454-f6109e4283da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:28.055042 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9066866-caf7-4500-9454-f6109e4283da admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:28.055798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9066866-caf7-4500-9454-f6109e4283da admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:28.056292 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9066866-caf7-4500-9454-f6109e4283da admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:28.057300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9066866-caf7-4500-9454-f6109e4283da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:28.059184 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9066866-caf7-4500-9454-f6109e4283da admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:28.072886 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 853/1713] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:28.077373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9066866-caf7-4500-9454-f6109e4283da admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:28.078928 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1194ada-8d1f-40b8-89e2-ed64191bf267 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:28.079494 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e9066866-caf7-4500-9454-f6109e4283da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:28.085006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6473844-c418-4c76-a2e5-48fec2fd2989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:28.086150 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6473844-c418-4c76-a2e5-48fec2fd2989 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:28.086684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6473844-c418-4c76-a2e5-48fec2fd2989 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:28.087181 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6473844-c418-4c76-a2e5-48fec2fd2989 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:28.088089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6473844-c418-4c76-a2e5-48fec2fd2989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:28.092417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6473844-c418-4c76-a2e5-48fec2fd2989 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:28.095746 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9066866-caf7-4500-9454-f6109e4283da admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:28.105447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 861/1714] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:28.107132 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6473844-c418-4c76-a2e5-48fec2fd2989 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:28.110505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57e313-ba4b-4abe-8df2-7f3f0665fd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:28.110708 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f6473844-c418-4c76-a2e5-48fec2fd2989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:28.111636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57e313-ba4b-4abe-8df2-7f3f0665fd06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:28.112161 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57e313-ba4b-4abe-8df2-7f3f0665fd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:28.112643 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57e313-ba4b-4abe-8df2-7f3f0665fd06 admin admin] PATH_INFO: `/v3/users/c5609fb7ffcf42d9b386763ea10a789c` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:28.115238 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57e313-ba4b-4abe-8df2-7f3f0665fd06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5609fb7ffcf42d9b386763ea10a789c)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:28.119515 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57e313-ba4b-4abe-8df2-7f3f0665fd06 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:28.127300 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6473844-c418-4c76-a2e5-48fec2fd2989 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:28.135164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2ac51-3ba4-4e5d-b369-02bbd38643f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:28.135837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2ac51-3ba4-4e5d-b369-02bbd38643f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:28.136122 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2ac51-3ba4-4e5d-b369-02bbd38643f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:28.136541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2ac51-3ba4-4e5d-b369-02bbd38643f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:28.137199 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2ac51-3ba4-4e5d-b369-02bbd38643f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:28.138974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2ac51-3ba4-4e5d-b369-02bbd38643f3 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:28.157891 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 862/1715] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:28.159070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2ac51-3ba4-4e5d-b369-02bbd38643f3 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:28.188049 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2ac51-3ba4-4e5d-b369-02bbd38643f3 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:28.196530 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b979f01-22ea-4de1-b5f4-57c5edb771a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:28.197698 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b979f01-22ea-4de1-b5f4-57c5edb771a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:28.198178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b979f01-22ea-4de1-b5f4-57c5edb771a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:28.198661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b979f01-22ea-4de1-b5f4-57c5edb771a7 admin admin] PATH_INFO: `/v3/projects/5f613cafcf984d5f8ca91ca26ba389f1/users/b976e32deafb422a83708c45f581e163/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:28.202809 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b979f01-22ea-4de1-b5f4-57c5edb771a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f613cafcf984d5f8ca91ca26ba389f1, user_id=b976e32deafb422a83708c45f581e163, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:28.209512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b979f01-22ea-4de1-b5f4-57c5edb771a7 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:28.237089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 863/1716] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:28 2022] PUT /identity/v3/projects/5f613cafcf984d5f8ca91ca26ba389f1/users/b976e32deafb422a83708c45f581e163/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:28.246656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b979f01-22ea-4de1-b5f4-57c5edb771a7 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:28.250235 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6f57e313-ba4b-4abe-8df2-7f3f0665fd06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5609fb7ffcf42d9b386763ea10a789c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:28.251846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6f57e313-ba4b-4abe-8df2-7f3f0665fd06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5609fb7ffcf42d9b386763ea10a789c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:28.252882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6f57e313-ba4b-4abe-8df2-7f3f0665fd06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5609fb7ffcf42d9b386763ea10a789c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:28.262896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:28.283537 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6f57e313-ba4b-4abe-8df2-7f3f0665fd06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5609fb7ffcf42d9b386763ea10a789c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:28.308611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6f57e313-ba4b-4abe-8df2-7f3f0665fd06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5609fb7ffcf42d9b386763ea10a789c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:28.336422 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 854/1717] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:28 2022] DELETE /identity/v3/users/c5609fb7ffcf42d9b386763ea10a789c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:28.338757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455004b-29e4-4876-be5d-4006c62a4871 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:28.339315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455004b-29e4-4876-be5d-4006c62a4871 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:28.339997 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455004b-29e4-4876-be5d-4006c62a4871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:28.348239 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b979f01-22ea-4de1-b5f4-57c5edb771a7 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:28.412193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e455004b-29e4-4876-be5d-4006c62a4871 None None] MFA Rules not processed for user `b976e32deafb422a83708c45f581e163`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:28.415429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e455004b-29e4-4876-be5d-4006c62a4871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:28.419454 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:28.460122 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70807492-e703-48b7-b4e1-685d95193939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:28.461443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70807492-e703-48b7-b4e1-685d95193939 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:28.461957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70807492-e703-48b7-b4e1-685d95193939 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:28.462438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70807492-e703-48b7-b4e1-685d95193939 admin admin] PATH_INFO: `/v3/projects/8839649043e14e588ab184df6e2252c8/users/75940736541747ccaf166649f1e9a3e8/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:28.496785 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70807492-e703-48b7-b4e1-685d95193939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8839649043e14e588ab184df6e2252c8, user_id=75940736541747ccaf166649f1e9a3e8, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:28.510347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70807492-e703-48b7-b4e1-685d95193939 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:28.555942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 864/1718] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:28 2022] PUT /identity/v3/projects/8839649043e14e588ab184df6e2252c8/users/75940736541747ccaf166649f1e9a3e8/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:28.569157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88f046-0847-49a1-96de-c7ea11d6c2e7 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:28.571028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88f046-0847-49a1-96de-c7ea11d6c2e7 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:28.571653 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88f046-0847-49a1-96de-c7ea11d6c2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:28.611431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 855/1719] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:28.629858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455004b-29e4-4876-be5d-4006c62a4871 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:28.632764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e455004b-29e4-4876-be5d-4006c62a4871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:28.636561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-bf88f046-0847-49a1-96de-c7ea11d6c2e7 None None] MFA Rules not processed for user `75940736541747ccaf166649f1e9a3e8`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:28.638969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-bf88f046-0847-49a1-96de-c7ea11d6c2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:28.758643 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455004b-29e4-4876-be5d-4006c62a4871 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:28.798458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 865/1720] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:28.816151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88f046-0847-49a1-96de-c7ea11d6c2e7 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:28.820914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-bf88f046-0847-49a1-96de-c7ea11d6c2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:28.901206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8410de-0c2e-42b6-95ef-ba8e2a90da9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:28.904374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8410de-0c2e-42b6-95ef-ba8e2a90da9c service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:28.906185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8410de-0c2e-42b6-95ef-ba8e2a90da9c service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:28.907174 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8410de-0c2e-42b6-95ef-ba8e2a90da9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:28.911508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ab8410de-0c2e-42b6-95ef-ba8e2a90da9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:28.980729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:29.007863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88f046-0847-49a1-96de-c7ea11d6c2e7 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:29.018109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b858690-e359-4e02-9d13-0b91eb86c038 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f5aabcb1414413a8dce7b76ce701582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:29.019102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b858690-e359-4e02-9d13-0b91eb86c038 service glance] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:29.019537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b858690-e359-4e02-9d13-0b91eb86c038 service glance] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:29.021005 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b858690-e359-4e02-9d13-0b91eb86c038 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:29.047988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b858690-e359-4e02-9d13-0b91eb86c038 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:29.061055 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b858690-e359-4e02-9d13-0b91eb86c038 service glance] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:29.070121 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8410de-0c2e-42b6-95ef-ba8e2a90da9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:29.073353 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8410de-0c2e-42b6-95ef-ba8e2a90da9c service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:29.094481 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 866/1721] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:29.099862 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 856/1722] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:50:29.444844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b858690-e359-4e02-9d13-0b91eb86c038 service glance] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:29.447817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1b858690-e359-4e02-9d13-0b91eb86c038 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:29.559392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:29.614423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b858690-e359-4e02-9d13-0b91eb86c038 service glance] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:29.624493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2466f30-7df7-4691-ac45-d8aca33ed23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:29.626317 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2466f30-7df7-4691-ac45-d8aca33ed23d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:29.627208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2466f30-7df7-4691-ac45-d8aca33ed23d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:29.628416 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2466f30-7df7-4691-ac45-d8aca33ed23d admin admin] PATH_INFO: `/v3/projects/406b33a302174fdaae923c02ec44d29a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:29.631393 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2466f30-7df7-4691-ac45-d8aca33ed23d admin admin] RBAC: Authorizing `identity:delete_project(project_id=406b33a302174fdaae923c02ec44d29a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:29.634134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2466f30-7df7-4691-ac45-d8aca33ed23d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:29.728808 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a2466f30-7df7-4691-ac45-d8aca33ed23d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '406b33a302174fdaae923c02ec44d29a', 'request_id': 'req-a2466f30-7df7-4691-ac45-d8aca33ed23d'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:29.749047 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a2466f30-7df7-4691-ac45-d8aca33ed23d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '406b33a302174fdaae923c02ec44d29a', 'request_id': 'req-a2466f30-7df7-4691-ac45-d8aca33ed23d'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:29.760899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a2466f30-7df7-4691-ac45-d8aca33ed23d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '406b33a302174fdaae923c02ec44d29a', 'request_id': 'req-a2466f30-7df7-4691-ac45-d8aca33ed23d'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:29.780951 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a2466f30-7df7-4691-ac45-d8aca33ed23d admin admin] The token cache is being invalidate because project 406b33a302174fdaae923c02ec44d29a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:29.781940 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a2466f30-7df7-4691-ac45-d8aca33ed23d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:29.783208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a2466f30-7df7-4691-ac45-d8aca33ed23d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:29.790073 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 867/1723] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:29 2022] DELETE /identity/v3/projects/406b33a302174fdaae923c02ec44d29a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:29.792273 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4ab0d-e6d0-4819-9ffa-bbd8157331a7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:29.793265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4ab0d-e6d0-4819-9ffa-bbd8157331a7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:29.793741 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4ab0d-e6d0-4819-9ffa-bbd8157331a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:29.830508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-67f4ab0d-e6d0-4819-9ffa-bbd8157331a7 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:29.832163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-67f4ab0d-e6d0-4819-9ffa-bbd8157331a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:29.975413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 857/1724] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:29.985003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eedfa-8a32-4cb4-9c13-85254d1740e9 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:29.986624 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eedfa-8a32-4cb4-9c13-85254d1740e9 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:29.986624 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eedfa-8a32-4cb4-9c13-85254d1740e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:30.018746 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-785eedfa-8a32-4cb4-9c13-85254d1740e9 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:30.020013 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-785eedfa-8a32-4cb4-9c13-85254d1740e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:30.052300 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 868/1725] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:30.053196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4ab0d-e6d0-4819-9ffa-bbd8157331a7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:30.072514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4ab0d-e6d0-4819-9ffa-bbd8157331a7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:30.089129 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcb65c-a0e9-42e0-b254-36f14a9e3d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:30.089771 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcb65c-a0e9-42e0-b254-36f14a9e3d07 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:30.090050 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcb65c-a0e9-42e0-b254-36f14a9e3d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:30.090342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcb65c-a0e9-42e0-b254-36f14a9e3d07 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:30.093019 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcb65c-a0e9-42e0-b254-36f14a9e3d07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:50:30.093387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcb65c-a0e9-42e0-b254-36f14a9e3d07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:30.095991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcb65c-a0e9-42e0-b254-36f14a9e3d07 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:30.130971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 858/1726] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:50:30.141585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785eedfa-8a32-4cb4-9c13-85254d1740e9 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:30.161569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785eedfa-8a32-4cb4-9c13-85254d1740e9 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:30.175742 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bee1d-45e9-4d25-a9fb-ecc7a2c5b0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:30.176726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bee1d-45e9-4d25-a9fb-ecc7a2c5b0f2 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:30.177195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bee1d-45e9-4d25-a9fb-ecc7a2c5b0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:30.177713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bee1d-45e9-4d25-a9fb-ecc7a2c5b0f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:30.178559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bee1d-45e9-4d25-a9fb-ecc7a2c5b0f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:30.181526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bee1d-45e9-4d25-a9fb-ecc7a2c5b0f2 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:30.227143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 869/1727] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:30 2022] POST /identity/v3/projects => generated 409 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:30.229394 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcb65c-a0e9-42e0-b254-36f14a9e3d07 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:30.250847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcb65c-a0e9-42e0-b254-36f14a9e3d07 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:30.324177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3bdd1-bf9a-4286-9f79-f4f4e2e213ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:30.324879 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3bdd1-bf9a-4286-9f79-f4f4e2e213ee admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:30.325143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3bdd1-bf9a-4286-9f79-f4f4e2e213ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:30.325555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3bdd1-bf9a-4286-9f79-f4f4e2e213ee admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:30.326221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3bdd1-bf9a-4286-9f79-f4f4e2e213ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:30.329311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3bdd1-bf9a-4286-9f79-f4f4e2e213ee admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:30.380432 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 859/1728] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:30 2022] POST /identity/v3/users => generated 305 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:30.381810 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bee1d-45e9-4d25-a9fb-ecc7a2c5b0f2 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:30.396582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bee1d-45e9-4d25-a9fb-ecc7a2c5b0f2 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:30.410223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc83b3-dbc1-4bc0-8c0c-6ac670d88ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:30.410863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc83b3-dbc1-4bc0-8c0c-6ac670d88ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:30.411138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc83b3-dbc1-4bc0-8c0c-6ac670d88ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:30.411403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc83b3-dbc1-4bc0-8c0c-6ac670d88ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:30.412095 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc83b3-dbc1-4bc0-8c0c-6ac670d88ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:30.413785 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc83b3-dbc1-4bc0-8c0c-6ac670d88ab4 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:30.442600 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 870/1729] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:30.444469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3bdd1-bf9a-4286-9f79-f4f4e2e213ee admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:30.463439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3bdd1-bf9a-4286-9f79-f4f4e2e213ee admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:30.494916 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866801f-6422-4b06-824b-c6e57fccb9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:30.495597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866801f-6422-4b06-824b-c6e57fccb9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:30.495907 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866801f-6422-4b06-824b-c6e57fccb9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:30.496174 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866801f-6422-4b06-824b-c6e57fccb9f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:30.496851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866801f-6422-4b06-824b-c6e57fccb9f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:30.499950 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866801f-6422-4b06-824b-c6e57fccb9f4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:30.551113 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 860/1730] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:30.552359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc83b3-dbc1-4bc0-8c0c-6ac670d88ab4 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:30.571249 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc83b3-dbc1-4bc0-8c0c-6ac670d88ab4 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:30.581380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb1575-b7e5-4ac2-8762-04c692c3aa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:30.582152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb1575-b7e5-4ac2-8762-04c692c3aa8a admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:30.582493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb1575-b7e5-4ac2-8762-04c692c3aa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:30.582827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb1575-b7e5-4ac2-8762-04c692c3aa8a admin admin] PATH_INFO: `/v3/projects/6ee8a2441ef54095a4768fd5d0c6567b/users/86ba8992097d493895a30655e03e9175/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:30.600030 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb1575-b7e5-4ac2-8762-04c692c3aa8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee8a2441ef54095a4768fd5d0c6567b, user_id=86ba8992097d493895a30655e03e9175, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:30.603615 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb1575-b7e5-4ac2-8762-04c692c3aa8a admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:30.622306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 871/1731] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:30 2022] PUT /identity/v3/projects/6ee8a2441ef54095a4768fd5d0c6567b/users/86ba8992097d493895a30655e03e9175/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:30.634912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40d483-1a08-4899-9cb6-448e08ccc54e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:30.635465 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40d483-1a08-4899-9cb6-448e08ccc54e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:30.636978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40d483-1a08-4899-9cb6-448e08ccc54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:30.680892 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-af40d483-1a08-4899-9cb6-448e08ccc54e None None] MFA Rules not processed for user `86ba8992097d493895a30655e03e9175`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:30.682150 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-af40d483-1a08-4899-9cb6-448e08ccc54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:30.703322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:30.766244 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 861/1732] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:30 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:30.787643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb1575-b7e5-4ac2-8762-04c692c3aa8a admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:30.912408 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb1575-b7e5-4ac2-8762-04c692c3aa8a admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:30.929885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9917c0-c808-4be3-b079-c02d47cb9afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:30.930523 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9917c0-c808-4be3-b079-c02d47cb9afd admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:30.930806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9917c0-c808-4be3-b079-c02d47cb9afd admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:30.931109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9917c0-c808-4be3-b079-c02d47cb9afd admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:30.931714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9917c0-c808-4be3-b079-c02d47cb9afd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:30.933672 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9917c0-c808-4be3-b079-c02d47cb9afd admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:30.944419 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:30.979396 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 872/1733] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:30 2022] POST /identity/v3/projects => generated 409 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:30.982754 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40d483-1a08-4899-9cb6-448e08ccc54e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:31.003317 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af40d483-1a08-4899-9cb6-448e08ccc54e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:31.096050 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1989b1-e80c-46ce-a5b1-7a90f588d2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:31.096873 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1989b1-e80c-46ce-a5b1-7a90f588d2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:31.097155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1989b1-e80c-46ce-a5b1-7a90f588d2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:31.097432 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1989b1-e80c-46ce-a5b1-7a90f588d2b2 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:31.098083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1989b1-e80c-46ce-a5b1-7a90f588d2b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:31.100311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1989b1-e80c-46ce-a5b1-7a90f588d2b2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:31.140119 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 862/1734] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:30 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:31.140243 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9917c0-c808-4be3-b079-c02d47cb9afd admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:31.153731 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9917c0-c808-4be3-b079-c02d47cb9afd admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:31.162807 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9559-4358-4eb3-84ff-a60f6167f9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:31.163337 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9559-4358-4eb3-84ff-a60f6167f9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:31.164482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9559-4358-4eb3-84ff-a60f6167f9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:31.164757 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9559-4358-4eb3-84ff-a60f6167f9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:31.165539 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e9559-4358-4eb3-84ff-a60f6167f9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:31.167748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e9559-4358-4eb3-84ff-a60f6167f9f8 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:31.189181 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 873/1735] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:31.194002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1989b1-e80c-46ce-a5b1-7a90f588d2b2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:31.209735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1989b1-e80c-46ce-a5b1-7a90f588d2b2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:31.216968 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60012b0-afdf-44b9-a230-c60fe885a94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:31.217671 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60012b0-afdf-44b9-a230-c60fe885a94a admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:31.217948 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60012b0-afdf-44b9-a230-c60fe885a94a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:31.218220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60012b0-afdf-44b9-a230-c60fe885a94a admin admin] PATH_INFO: `/v3/projects/45985ce963c44a6185f661ce438f2c24/users/615a21f4a44a49ba8c2ebee45ef2c584/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:31.245812 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60012b0-afdf-44b9-a230-c60fe885a94a admin admin] RBAC: Authorizing `identity:create_grant(project_id=45985ce963c44a6185f661ce438f2c24, user_id=615a21f4a44a49ba8c2ebee45ef2c584, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:31.249546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60012b0-afdf-44b9-a230-c60fe885a94a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:31.266380 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 863/1736] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:31 2022] PUT /identity/v3/projects/45985ce963c44a6185f661ce438f2c24/users/615a21f4a44a49ba8c2ebee45ef2c584/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:50:31.270854 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9559-4358-4eb3-84ff-a60f6167f9f8 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:31.288422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9559-4358-4eb3-84ff-a60f6167f9f8 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:31.386645 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fd985-ad95-4956-a96f-84adfd6c3f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:31.387268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fd985-ad95-4956-a96f-84adfd6c3f34 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:31.387828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fd985-ad95-4956-a96f-84adfd6c3f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:31.388320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fd985-ad95-4956-a96f-84adfd6c3f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:31.389035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088fd985-ad95-4956-a96f-84adfd6c3f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:31.390680 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088fd985-ad95-4956-a96f-84adfd6c3f34 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:31.401726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 874/1737] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:31.406946 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60012b0-afdf-44b9-a230-c60fe885a94a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:31.417992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60012b0-afdf-44b9-a230-c60fe885a94a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:31.423610 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4860ee3-51d7-4a0d-9b29-46e235ac626a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:31.424365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4860ee3-51d7-4a0d-9b29-46e235ac626a admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:31.424717 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4860ee3-51d7-4a0d-9b29-46e235ac626a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:31.424884 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4860ee3-51d7-4a0d-9b29-46e235ac626a admin admin] PATH_INFO: `/v3/projects/45985ce963c44a6185f661ce438f2c24/users/615a21f4a44a49ba8c2ebee45ef2c584/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:31.426865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4860ee3-51d7-4a0d-9b29-46e235ac626a admin admin] RBAC: Authorizing `identity:create_grant(project_id=45985ce963c44a6185f661ce438f2c24, user_id=615a21f4a44a49ba8c2ebee45ef2c584, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:31.429782 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4860ee3-51d7-4a0d-9b29-46e235ac626a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:31.453733 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 864/1738] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:31 2022] PUT /identity/v3/projects/45985ce963c44a6185f661ce438f2c24/users/615a21f4a44a49ba8c2ebee45ef2c584/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:50:31.464596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9a9f0-d1ed-45c1-808b-12eb68c72b89 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:31.464890 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9a9f0-d1ed-45c1-808b-12eb68c72b89 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:31.465198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9a9f0-d1ed-45c1-808b-12eb68c72b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:31.507572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-f0b9a9f0-d1ed-45c1-808b-12eb68c72b89 None None] MFA Rules not processed for user `615a21f4a44a49ba8c2ebee45ef2c584`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:31.508839 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f0b9a9f0-d1ed-45c1-808b-12eb68c72b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:31.635596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 875/1739] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:31 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:31.650189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4860ee3-51d7-4a0d-9b29-46e235ac626a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:31.652736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c4860ee3-51d7-4a0d-9b29-46e235ac626a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:31.730017 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4860ee3-51d7-4a0d-9b29-46e235ac626a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:31.734900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302639e-be43-4d67-a68a-75f6dcf6de58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:31.735575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302639e-be43-4d67-a68a-75f6dcf6de58 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:31.735905 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302639e-be43-4d67-a68a-75f6dcf6de58 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:31.736182 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302639e-be43-4d67-a68a-75f6dcf6de58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:31.757138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302639e-be43-4d67-a68a-75f6dcf6de58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:31.761046 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302639e-be43-4d67-a68a-75f6dcf6de58 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:31.774070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 865/1740] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:31 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:50:31.984544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9a9f0-d1ed-45c1-808b-12eb68c72b89 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:31.996233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9a9f0-d1ed-45c1-808b-12eb68c72b89 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:32.000188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a922de-ef42-4527-a3b6-ff8e423e26fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:32.000847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a922de-ef42-4527-a3b6-ff8e423e26fb service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:32.001163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a922de-ef42-4527-a3b6-ff8e423e26fb service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:32.001443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a922de-ef42-4527-a3b6-ff8e423e26fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:32.071469 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a922de-ef42-4527-a3b6-ff8e423e26fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:32.073491 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a922de-ef42-4527-a3b6-ff8e423e26fb service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:32.077154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:32.088754 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 876/1741] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:31 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:40.776713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302639e-be43-4d67-a68a-75f6dcf6de58 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:40.786292 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302639e-be43-4d67-a68a-75f6dcf6de58 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:40.791024 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd21ee-8a39-49aa-bc5e-677d4a0a3e98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:40.791615 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd21ee-8a39-49aa-bc5e-677d4a0a3e98 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:40.791810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd21ee-8a39-49aa-bc5e-677d4a0a3e98 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:40.792106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd21ee-8a39-49aa-bc5e-677d4a0a3e98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:40.794836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d7fd21ee-8a39-49aa-bc5e-677d4a0a3e98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:40.809661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:40.897316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd21ee-8a39-49aa-bc5e-677d4a0a3e98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:40.898967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd21ee-8a39-49aa-bc5e-677d4a0a3e98 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:40.910480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 866/1742] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:42.085990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:43.277721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a922de-ef42-4527-a3b6-ff8e423e26fb service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:43.385711 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a922de-ef42-4527-a3b6-ff8e423e26fb service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:43.393786 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896edd0e-d2e0-4764-986f-5c4b7c9163f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:43.400520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896edd0e-d2e0-4764-986f-5c4b7c9163f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:43.401309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896edd0e-d2e0-4764-986f-5c4b7c9163f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:43.405004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896edd0e-d2e0-4764-986f-5c4b7c9163f2 admin admin] PATH_INFO: `/v3/users/86ba8992097d493895a30655e03e9175` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:43.408793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896edd0e-d2e0-4764-986f-5c4b7c9163f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86ba8992097d493895a30655e03e9175)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:43.410845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896edd0e-d2e0-4764-986f-5c4b7c9163f2 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:43.536403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-896edd0e-d2e0-4764-986f-5c4b7c9163f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86ba8992097d493895a30655e03e9175'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:43.559146 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-896edd0e-d2e0-4764-986f-5c4b7c9163f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86ba8992097d493895a30655e03e9175'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:43.568979 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-896edd0e-d2e0-4764-986f-5c4b7c9163f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86ba8992097d493895a30655e03e9175'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:43.576563 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-896edd0e-d2e0-4764-986f-5c4b7c9163f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86ba8992097d493895a30655e03e9175'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:43.591024 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-896edd0e-d2e0-4764-986f-5c4b7c9163f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86ba8992097d493895a30655e03e9175'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:43.626136 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 877/1743] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:43 2022] DELETE /identity/v3/users/86ba8992097d493895a30655e03e9175 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:43.631816 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343aa7f0-9949-4c6c-ba40-554a4b5339a7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:43.632417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343aa7f0-9949-4c6c-ba40-554a4b5339a7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:43.632417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343aa7f0-9949-4c6c-ba40-554a4b5339a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:43.672975 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-343aa7f0-9949-4c6c-ba40-554a4b5339a7 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:43.675047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-343aa7f0-9949-4c6c-ba40-554a4b5339a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:43.883937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 867/1744] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:43.899264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896edd0e-d2e0-4764-986f-5c4b7c9163f2 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:43.902245 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-896edd0e-d2e0-4764-986f-5c4b7c9163f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:44.125185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896edd0e-d2e0-4764-986f-5c4b7c9163f2 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:44.139868 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712531e-fdc1-4f33-b359-7b4fcbd35cb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:44.140612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712531e-fdc1-4f33-b359-7b4fcbd35cb5 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:44.140889 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712531e-fdc1-4f33-b359-7b4fcbd35cb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:44.141161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712531e-fdc1-4f33-b359-7b4fcbd35cb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:44.166569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712531e-fdc1-4f33-b359-7b4fcbd35cb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:44.169495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712531e-fdc1-4f33-b359-7b4fcbd35cb5 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:44.197424 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 878/1745] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:44.764366 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343aa7f0-9949-4c6c-ba40-554a4b5339a7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:44.768194 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-343aa7f0-9949-4c6c-ba40-554a4b5339a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:44.807611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343aa7f0-9949-4c6c-ba40-554a4b5339a7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:44.839411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:44.840660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:44.841408 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:44.841908 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] PATH_INFO: `/v3/projects/6ee8a2441ef54095a4768fd5d0c6567b` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:44.853503 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ee8a2441ef54095a4768fd5d0c6567b)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:44.855688 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:44.978285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ee8a2441ef54095a4768fd5d0c6567b', 'request_id': 'req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:44.989075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ee8a2441ef54095a4768fd5d0c6567b', 'request_id': 'req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:45.001933 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ee8a2441ef54095a4768fd5d0c6567b', 'request_id': 'req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:45.023673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] The token cache is being invalidate because project 6ee8a2441ef54095a4768fd5d0c6567b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:45.024168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:45.025158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:45.027688 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 868/1746] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:44 2022] DELETE /identity/v3/projects/6ee8a2441ef54095a4768fd5d0c6567b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:45.032824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712531e-fdc1-4f33-b359-7b4fcbd35cb5 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:45.048935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712531e-fdc1-4f33-b359-7b4fcbd35cb5 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:45.072685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:45.149100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b575d-1f9f-49f5-a9e6-3ff33dfb2b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:45.149799 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b575d-1f9f-49f5-a9e6-3ff33dfb2b86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:45.150069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b575d-1f9f-49f5-a9e6-3ff33dfb2b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:45.150537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b575d-1f9f-49f5-a9e6-3ff33dfb2b86 admin admin] PATH_INFO: `/v3/users/615a21f4a44a49ba8c2ebee45ef2c584` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:45.151841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b575d-1f9f-49f5-a9e6-3ff33dfb2b86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=615a21f4a44a49ba8c2ebee45ef2c584)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:45.153849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b575d-1f9f-49f5-a9e6-3ff33dfb2b86 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:45.260469 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-102b575d-1f9f-49f5-a9e6-3ff33dfb2b86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '615a21f4a44a49ba8c2ebee45ef2c584'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:45.270421 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-102b575d-1f9f-49f5-a9e6-3ff33dfb2b86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '615a21f4a44a49ba8c2ebee45ef2c584'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:45.272447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-102b575d-1f9f-49f5-a9e6-3ff33dfb2b86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '615a21f4a44a49ba8c2ebee45ef2c584'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:45.274157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-102b575d-1f9f-49f5-a9e6-3ff33dfb2b86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '615a21f4a44a49ba8c2ebee45ef2c584'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:45.293901 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-102b575d-1f9f-49f5-a9e6-3ff33dfb2b86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '615a21f4a44a49ba8c2ebee45ef2c584'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:45.328035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 879/1747] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:45 2022] DELETE /identity/v3/users/615a21f4a44a49ba8c2ebee45ef2c584 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:46.075405 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:46.077796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:46.118317 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0604a-62f7-4c3d-a6a0-468a76f5fe21 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:46.174728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:46.220223 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565dc6d8-d302-4a4c-92c5-acc529b7eadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:46.221257 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565dc6d8-d302-4a4c-92c5-acc529b7eadc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:46.221633 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565dc6d8-d302-4a4c-92c5-acc529b7eadc admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:46.222003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565dc6d8-d302-4a4c-92c5-acc529b7eadc admin admin] PATH_INFO: `/v3/projects/45985ce963c44a6185f661ce438f2c24` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:46.224297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565dc6d8-d302-4a4c-92c5-acc529b7eadc admin admin] RBAC: Authorizing `identity:delete_project(project_id=45985ce963c44a6185f661ce438f2c24)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:46.228385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565dc6d8-d302-4a4c-92c5-acc529b7eadc admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:46.294066 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-565dc6d8-d302-4a4c-92c5-acc529b7eadc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45985ce963c44a6185f661ce438f2c24', 'request_id': 'req-565dc6d8-d302-4a4c-92c5-acc529b7eadc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:46.301147 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-565dc6d8-d302-4a4c-92c5-acc529b7eadc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45985ce963c44a6185f661ce438f2c24', 'request_id': 'req-565dc6d8-d302-4a4c-92c5-acc529b7eadc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:46.313324 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-565dc6d8-d302-4a4c-92c5-acc529b7eadc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45985ce963c44a6185f661ce438f2c24', 'request_id': 'req-565dc6d8-d302-4a4c-92c5-acc529b7eadc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:46.326540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-565dc6d8-d302-4a4c-92c5-acc529b7eadc admin admin] The token cache is being invalidate because project 45985ce963c44a6185f661ce438f2c24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:46.326983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-565dc6d8-d302-4a4c-92c5-acc529b7eadc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:46.327957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-565dc6d8-d302-4a4c-92c5-acc529b7eadc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:46.330073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 869/1748] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:46 2022] DELETE /identity/v3/projects/45985ce963c44a6185f661ce438f2c24 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:46.334957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063f4a5-3728-4603-9f5f-0cd16354b1c2 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:46.336187 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063f4a5-3728-4603-9f5f-0cd16354b1c2 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:46.338403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063f4a5-3728-4603-9f5f-0cd16354b1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:46.373258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-4063f4a5-3728-4603-9f5f-0cd16354b1c2 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:46.374644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4063f4a5-3728-4603-9f5f-0cd16354b1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:46.533688 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 880/1749] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:46.541556 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c94d7-77d3-4097-9230-5a1286bc7438 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:46.541933 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c94d7-77d3-4097-9230-5a1286bc7438 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:46.542096 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c94d7-77d3-4097-9230-5a1286bc7438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:46.590847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-482c94d7-77d3-4097-9230-5a1286bc7438 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:46.592197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-482c94d7-77d3-4097-9230-5a1286bc7438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:46.605008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 870/1750] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:50:46.606610 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063f4a5-3728-4603-9f5f-0cd16354b1c2 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:46.619088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063f4a5-3728-4603-9f5f-0cd16354b1c2 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:46.624438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cd485-0f35-4988-9afc-f67fdfc9a791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:46.625371 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cd485-0f35-4988-9afc-f67fdfc9a791 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:46.625720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cd485-0f35-4988-9afc-f67fdfc9a791 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:46.626056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cd485-0f35-4988-9afc-f67fdfc9a791 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:46.626806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2cd485-0f35-4988-9afc-f67fdfc9a791 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:50:46.627175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2cd485-0f35-4988-9afc-f67fdfc9a791 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:46.629719 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2cd485-0f35-4988-9afc-f67fdfc9a791 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:46.640230 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c94d7-77d3-4097-9230-5a1286bc7438 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:46.642566 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-482c94d7-77d3-4097-9230-5a1286bc7438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:46.658603 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c94d7-77d3-4097-9230-5a1286bc7438 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:46.672471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 881/1751] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:50:46.674917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cd485-0f35-4988-9afc-f67fdfc9a791 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:46.683682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e6d17-ef5b-44ba-93d3-c9d3fb849f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:46.685391 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e6d17-ef5b-44ba-93d3-c9d3fb849f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:46.686421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e6d17-ef5b-44ba-93d3-c9d3fb849f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:46.687249 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e6d17-ef5b-44ba-93d3-c9d3fb849f0c admin admin] PATH_INFO: `/v3/users/2c041502ff4f43eda21d905cc11e1229` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:46.690475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e6d17-ef5b-44ba-93d3-c9d3fb849f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c041502ff4f43eda21d905cc11e1229)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:46.694123 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cd485-0f35-4988-9afc-f67fdfc9a791 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:46.696024 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e6d17-ef5b-44ba-93d3-c9d3fb849f0c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:46.700363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a8e7a-8433-4b53-8b6d-55f556c79064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:46.701090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a8e7a-8433-4b53-8b6d-55f556c79064 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:46.701431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a8e7a-8433-4b53-8b6d-55f556c79064 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:46.701762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a8e7a-8433-4b53-8b6d-55f556c79064 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:46.703183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a8e7a-8433-4b53-8b6d-55f556c79064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:46.705859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a8e7a-8433-4b53-8b6d-55f556c79064 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:46.768701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 882/1752] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:46 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:46.771649 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a8e7a-8433-4b53-8b6d-55f556c79064 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:46.786086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0d3e6d17-ef5b-44ba-93d3-c9d3fb849f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c041502ff4f43eda21d905cc11e1229'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:46.787618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0d3e6d17-ef5b-44ba-93d3-c9d3fb849f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c041502ff4f43eda21d905cc11e1229'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:46.791431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0d3e6d17-ef5b-44ba-93d3-c9d3fb849f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c041502ff4f43eda21d905cc11e1229'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:46.805040 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:46.820582 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0d3e6d17-ef5b-44ba-93d3-c9d3fb849f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c041502ff4f43eda21d905cc11e1229'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:46.839714 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0d3e6d17-ef5b-44ba-93d3-c9d3fb849f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c041502ff4f43eda21d905cc11e1229'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:46.887937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 871/1753] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:46 2022] DELETE /identity/v3/users/2c041502ff4f43eda21d905cc11e1229 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:46.894648 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a83c61-c1ce-4a05-afd4-4aa4dd9e8d9e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:46.895298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a83c61-c1ce-4a05-afd4-4aa4dd9e8d9e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:46.896644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a83c61-c1ce-4a05-afd4-4aa4dd9e8d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:46.951567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c4a83c61-c1ce-4a05-afd4-4aa4dd9e8d9e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:46.953966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c4a83c61-c1ce-4a05-afd4-4aa4dd9e8d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:46.983111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a8e7a-8433-4b53-8b6d-55f556c79064 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:47.034060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:47.131308 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c077f-d126-43dd-b0f2-a4658c2d82d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:47.132847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c077f-d126-43dd-b0f2-a4658c2d82d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:47.133296 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c077f-d126-43dd-b0f2-a4658c2d82d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:47.133778 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c077f-d126-43dd-b0f2-a4658c2d82d6 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:47.134498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c077f-d126-43dd-b0f2-a4658c2d82d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:47.137765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c077f-d126-43dd-b0f2-a4658c2d82d6 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:47.144605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 872/1754] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:47.157970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a83c61-c1ce-4a05-afd4-4aa4dd9e8d9e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:47.161009 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c4a83c61-c1ce-4a05-afd4-4aa4dd9e8d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:47.203140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 883/1755] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:46 2022] POST /identity/v3/users => generated 293 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:47.209141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c077f-d126-43dd-b0f2-a4658c2d82d6 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:47.212408 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c13c077f-d126-43dd-b0f2-a4658c2d82d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:47.236911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c077f-d126-43dd-b0f2-a4658c2d82d6 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:47.245762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796679c3-dd92-4401-9b03-f8d05ba836e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:47.247071 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796679c3-dd92-4401-9b03-f8d05ba836e7 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:47.247352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796679c3-dd92-4401-9b03-f8d05ba836e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:47.248608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796679c3-dd92-4401-9b03-f8d05ba836e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:47.248608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796679c3-dd92-4401-9b03-f8d05ba836e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:47.252016 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796679c3-dd92-4401-9b03-f8d05ba836e7 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:47.272659 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 884/1756] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:47.277016 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796679c3-dd92-4401-9b03-f8d05ba836e7 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:47.289229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796679c3-dd92-4401-9b03-f8d05ba836e7 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:47.317608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd97e11-77c2-4baf-b40e-6e94c0443b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:47.318795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd97e11-77c2-4baf-b40e-6e94c0443b03 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:47.320021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd97e11-77c2-4baf-b40e-6e94c0443b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:47.320519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd97e11-77c2-4baf-b40e-6e94c0443b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:47.321866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd97e11-77c2-4baf-b40e-6e94c0443b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:47.327986 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd97e11-77c2-4baf-b40e-6e94c0443b03 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:47.345420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a83c61-c1ce-4a05-afd4-4aa4dd9e8d9e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:47.349348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 885/1757] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:47.353808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abad48-90f1-4bb7-a4f3-f11e4e4f693d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:47.354458 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abad48-90f1-4bb7-a4f3-f11e4e4f693d service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:47.354757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abad48-90f1-4bb7-a4f3-f11e4e4f693d service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:47.354990 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abad48-90f1-4bb7-a4f3-f11e4e4f693d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:47.358117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd97e11-77c2-4baf-b40e-6e94c0443b03 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:47.370973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abad48-90f1-4bb7-a4f3-f11e4e4f693d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:47.372725 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abad48-90f1-4bb7-a4f3-f11e4e4f693d service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:47.383089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd97e11-77c2-4baf-b40e-6e94c0443b03 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:47.386365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 873/1758] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:50:47.391760 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e3249-5a92-4467-9a5d-9e924fa1c1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:47.393328 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e3249-5a92-4467-9a5d-9e924fa1c1b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:47.394573 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e3249-5a92-4467-9a5d-9e924fa1c1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:47.395174 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e3249-5a92-4467-9a5d-9e924fa1c1b1 admin admin] PATH_INFO: `/v3/projects/8c0683d58cdb43408f408d9301909caf/users/8081f0a1c2e14ab296052a79aa0ae7a3/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:47.429836 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e3249-5a92-4467-9a5d-9e924fa1c1b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0683d58cdb43408f408d9301909caf, user_id=8081f0a1c2e14ab296052a79aa0ae7a3, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:47.437520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e3249-5a92-4467-9a5d-9e924fa1c1b1 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:47.463388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 886/1759] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:47 2022] PUT /identity/v3/projects/8c0683d58cdb43408f408d9301909caf/users/8081f0a1c2e14ab296052a79aa0ae7a3/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:50:47.486724 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5746e8b-6f6d-43e0-a8ac-414a73c9183d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:47.487205 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5746e8b-6f6d-43e0-a8ac-414a73c9183d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:47.487579 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5746e8b-6f6d-43e0-a8ac-414a73c9183d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:47.528327 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b5746e8b-6f6d-43e0-a8ac-414a73c9183d None None] MFA Rules not processed for user `8081f0a1c2e14ab296052a79aa0ae7a3`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:47.529728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b5746e8b-6f6d-43e0-a8ac-414a73c9183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:47.665504 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e3249-5a92-4467-9a5d-9e924fa1c1b1 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:47.667901 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 874/1760] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:47.772641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:47.780186 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e3249-5a92-4467-9a5d-9e924fa1c1b1 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:47.787768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474210c-b8ac-4a91-bdee-9fd5f6463601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:47.788550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474210c-b8ac-4a91-bdee-9fd5f6463601 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:47.788848 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474210c-b8ac-4a91-bdee-9fd5f6463601 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:47.789123 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474210c-b8ac-4a91-bdee-9fd5f6463601 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:47.789776 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474210c-b8ac-4a91-bdee-9fd5f6463601 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:47.791788 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474210c-b8ac-4a91-bdee-9fd5f6463601 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:47.826979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5746e8b-6f6d-43e0-a8ac-414a73c9183d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:47.831799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b5746e8b-6f6d-43e0-a8ac-414a73c9183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:47.857383 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 887/1761] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:47 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:47.859127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474210c-b8ac-4a91-bdee-9fd5f6463601 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:47.864721 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5746e8b-6f6d-43e0-a8ac-414a73c9183d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:47.948113 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f08695-4732-4190-b978-81211a80e2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:47.948908 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f08695-4732-4190-b978-81211a80e2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:47.949250 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f08695-4732-4190-b978-81211a80e2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:47.949586 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f08695-4732-4190-b978-81211a80e2b8 admin admin] PATH_INFO: `/v3/users/bc29734859a5460faee189b1726b50b4` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:47.950865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f08695-4732-4190-b978-81211a80e2b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc29734859a5460faee189b1726b50b4)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:47.953316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f08695-4732-4190-b978-81211a80e2b8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:47.961376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:47.964011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474210c-b8ac-4a91-bdee-9fd5f6463601 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:47.969163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48923d89-8e33-4bfd-921c-779b682c0836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:47.969847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48923d89-8e33-4bfd-921c-779b682c0836 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:47.970116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48923d89-8e33-4bfd-921c-779b682c0836 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:47.970392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48923d89-8e33-4bfd-921c-779b682c0836 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:47.971921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48923d89-8e33-4bfd-921c-779b682c0836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:47.973895 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48923d89-8e33-4bfd-921c-779b682c0836 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:48.001844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 888/1762] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:47 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:48.006576 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48923d89-8e33-4bfd-921c-779b682c0836 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:48.019951 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48923d89-8e33-4bfd-921c-779b682c0836 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:48.025548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ff3af-0499-4156-9018-606695c72167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:48.026446 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff3af-0499-4156-9018-606695c72167 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:48.026734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff3af-0499-4156-9018-606695c72167 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:48.027105 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff3af-0499-4156-9018-606695c72167 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:48.028244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ff3af-0499-4156-9018-606695c72167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:48.030647 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ff3af-0499-4156-9018-606695c72167 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:48.036691 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97f08695-4732-4190-b978-81211a80e2b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc29734859a5460faee189b1726b50b4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.038216 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97f08695-4732-4190-b978-81211a80e2b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc29734859a5460faee189b1726b50b4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.039194 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97f08695-4732-4190-b978-81211a80e2b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc29734859a5460faee189b1726b50b4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.047922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 889/1763] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:48.049347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ff3af-0499-4156-9018-606695c72167 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:48.051108 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-fe7ff3af-0499-4156-9018-606695c72167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:48.069313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97f08695-4732-4190-b978-81211a80e2b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc29734859a5460faee189b1726b50b4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.070166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ff3af-0499-4156-9018-606695c72167 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:48.078145 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d340d-f06b-45b5-87ca-6d851ff89906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:48.079107 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d340d-f06b-45b5-87ca-6d851ff89906 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:48.079552 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d340d-f06b-45b5-87ca-6d851ff89906 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:48.080096 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d340d-f06b-45b5-87ca-6d851ff89906 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:48.081245 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d340d-f06b-45b5-87ca-6d851ff89906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:48.084244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d340d-f06b-45b5-87ca-6d851ff89906 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:48.084813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97f08695-4732-4190-b978-81211a80e2b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc29734859a5460faee189b1726b50b4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.106204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 890/1764] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:48.108636 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d340d-f06b-45b5-87ca-6d851ff89906 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:48.112584 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 875/1765] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:47 2022] DELETE /identity/v3/users/bc29734859a5460faee189b1726b50b4 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:50:48.126144 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d340d-f06b-45b5-87ca-6d851ff89906 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:48.243437 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35920836-195a-466c-92c3-5b679cbf9444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:48.244474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35920836-195a-466c-92c3-5b679cbf9444 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:48.244753 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35920836-195a-466c-92c3-5b679cbf9444 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:48.245025 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35920836-195a-466c-92c3-5b679cbf9444 admin admin] PATH_INFO: `/v3/projects/4662a487309e4da29cdb223ea2254a4d/users/8a5dcdad9c804abda02c0ba66e45d5e2/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:48.266756 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35920836-195a-466c-92c3-5b679cbf9444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4662a487309e4da29cdb223ea2254a4d, user_id=8a5dcdad9c804abda02c0ba66e45d5e2, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:48.269606 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35920836-195a-466c-92c3-5b679cbf9444 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:48.277337 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f08695-4732-4190-b978-81211a80e2b8 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:48.279542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-97f08695-4732-4190-b978-81211a80e2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:48.291688 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 891/1766] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:48 2022] PUT /identity/v3/projects/4662a487309e4da29cdb223ea2254a4d/users/8a5dcdad9c804abda02c0ba66e45d5e2/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:48.294673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f08695-4732-4190-b978-81211a80e2b8 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:48.302943 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c542ea-e402-45b0-818f-6eb7240497ab None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:48.303384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c542ea-e402-45b0-818f-6eb7240497ab None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:48.303655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c542ea-e402-45b0-818f-6eb7240497ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:48.365405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-38c542ea-e402-45b0-818f-6eb7240497ab None None] MFA Rules not processed for user `8a5dcdad9c804abda02c0ba66e45d5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:48.366649 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-38c542ea-e402-45b0-818f-6eb7240497ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:48.424876 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:48.443509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:48.444583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:48.444905 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:48.445178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] PATH_INFO: `/v3/projects/15c81d35884a48a0925d7e8ce7d6df22` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:48.446522 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] RBAC: Authorizing `identity:delete_project(project_id=15c81d35884a48a0925d7e8ce7d6df22)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:48.448682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:48.511527 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 892/1767] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:48.517779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c542ea-e402-45b0-818f-6eb7240497ab None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:48.518541 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15c81d35884a48a0925d7e8ce7d6df22', 'request_id': 'req-af4764d4-6ea1-485e-bfca-b05718edf24a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.533896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15c81d35884a48a0925d7e8ce7d6df22', 'request_id': 'req-af4764d4-6ea1-485e-bfca-b05718edf24a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.540405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c542ea-e402-45b0-818f-6eb7240497ab None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:48.544376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15c81d35884a48a0925d7e8ce7d6df22', 'request_id': 'req-af4764d4-6ea1-485e-bfca-b05718edf24a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.559710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] The token cache is being invalidate because project 15c81d35884a48a0925d7e8ce7d6df22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:48.560220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.561064 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.563552 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 876/1768] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:48 2022] DELETE /identity/v3/projects/15c81d35884a48a0925d7e8ce7d6df22 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:48.567749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:48.569230 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:48.585356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:48.672227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa6864-309c-4e44-812d-db9bcb22f6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:48.673466 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa6864-309c-4e44-812d-db9bcb22f6e0 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:48.674146 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa6864-309c-4e44-812d-db9bcb22f6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:48.674556 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa6864-309c-4e44-812d-db9bcb22f6e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:48.675823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa6864-309c-4e44-812d-db9bcb22f6e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:48.678350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4764d4-6ea1-485e-bfca-b05718edf24a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:48.680502 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa6864-309c-4e44-812d-db9bcb22f6e0 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:48.693003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56ca1d-e261-411b-99ba-cf34272f71f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:48.694180 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56ca1d-e261-411b-99ba-cf34272f71f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:48.694501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56ca1d-e261-411b-99ba-cf34272f71f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:48.694781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56ca1d-e261-411b-99ba-cf34272f71f7 admin admin] PATH_INFO: `/v3/users/b976e32deafb422a83708c45f581e163` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:48.696158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56ca1d-e261-411b-99ba-cf34272f71f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b976e32deafb422a83708c45f581e163)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:48.699567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56ca1d-e261-411b-99ba-cf34272f71f7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:48.763104 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 893/1769] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:48 2022] POST /identity/v3/projects => generated 385 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:48.770105 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa6864-309c-4e44-812d-db9bcb22f6e0 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:48.777712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-21fa6864-309c-4e44-812d-db9bcb22f6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:48.815217 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ba56ca1d-e261-411b-99ba-cf34272f71f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b976e32deafb422a83708c45f581e163'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.816078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ba56ca1d-e261-411b-99ba-cf34272f71f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b976e32deafb422a83708c45f581e163'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.816823 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ba56ca1d-e261-411b-99ba-cf34272f71f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b976e32deafb422a83708c45f581e163'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.839597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ba56ca1d-e261-411b-99ba-cf34272f71f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b976e32deafb422a83708c45f581e163'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.852269 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ba56ca1d-e261-411b-99ba-cf34272f71f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b976e32deafb422a83708c45f581e163'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:48.872587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 877/1770] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:50:48 2022] DELETE /identity/v3/users/b976e32deafb422a83708c45f581e163 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:50:48.877665 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56ca1d-e261-411b-99ba-cf34272f71f7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:48.877665 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ba56ca1d-e261-411b-99ba-cf34272f71f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:48.961872 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa6864-309c-4e44-812d-db9bcb22f6e0 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:49.017057 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56ca1d-e261-411b-99ba-cf34272f71f7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:49.024114 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80b346-2234-4072-8fac-0ba1ccdbd715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:49.024662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80b346-2234-4072-8fac-0ba1ccdbd715 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.024979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80b346-2234-4072-8fac-0ba1ccdbd715 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.025547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80b346-2234-4072-8fac-0ba1ccdbd715 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.026339 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80b346-2234-4072-8fac-0ba1ccdbd715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:49.029064 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80b346-2234-4072-8fac-0ba1ccdbd715 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:49.054420 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:49.059674 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 878/1771] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:48 2022] POST /identity/v3/users => generated 299 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:49.061338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80b346-2234-4072-8fac-0ba1ccdbd715 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:49.063976 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:49.064746 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.065239 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.065582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9 admin admin] PATH_INFO: `/v3/projects/984a02b8dfd446879438d29b5efdd996` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.067535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=984a02b8dfd446879438d29b5efdd996)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:49.069650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:49.074526 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80b346-2234-4072-8fac-0ba1ccdbd715 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:49.099302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe77be-3411-4961-b01d-b5c738b77b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:49.099974 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe77be-3411-4961-b01d-b5c738b77b58 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.100253 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe77be-3411-4961-b01d-b5c738b77b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.103591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe77be-3411-4961-b01d-b5c738b77b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.104341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe77be-3411-4961-b01d-b5c738b77b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:49.109190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe77be-3411-4961-b01d-b5c738b77b58 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:49.130415 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 879/1772] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:49.132765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe77be-3411-4961-b01d-b5c738b77b58 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:49.146841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '984a02b8dfd446879438d29b5efdd996', 'request_id': 'req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:49.149334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe77be-3411-4961-b01d-b5c738b77b58 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:49.159160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '984a02b8dfd446879438d29b5efdd996', 'request_id': 'req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:49.159449 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:49.170889 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '984a02b8dfd446879438d29b5efdd996', 'request_id': 'req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:49.185333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9 admin admin] The token cache is being invalidate because project 984a02b8dfd446879438d29b5efdd996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:49.185715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:49.186583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5b3c0b7b-3d2c-4e6b-b27a-1657849a4cc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:49.194113 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 894/1773] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:48 2022] DELETE /identity/v3/projects/984a02b8dfd446879438d29b5efdd996 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:49.196131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3937a5-09a2-4288-be1d-884a60c18192 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.196414 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3937a5-09a2-4288-be1d-884a60c18192 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.196878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3937a5-09a2-4288-be1d-884a60c18192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.223424 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-3c3937a5-09a2-4288-be1d-884a60c18192 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:49.224962 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3c3937a5-09a2-4288-be1d-884a60c18192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:49.270349 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4607a09-8058-4c58-8f6a-140f7756d32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:49.271713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4607a09-8058-4c58-8f6a-140f7756d32d admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.272089 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4607a09-8058-4c58-8f6a-140f7756d32d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.272370 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4607a09-8058-4c58-8f6a-140f7756d32d admin admin] PATH_INFO: `/v3/projects/0ca507385d134b47aa9a588f8c5c1170/users/2e5f68ae260e44abba5d0d47647a9cd0/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.293458 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4607a09-8058-4c58-8f6a-140f7756d32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca507385d134b47aa9a588f8c5c1170, user_id=2e5f68ae260e44abba5d0d47647a9cd0, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:49.297002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4607a09-8058-4c58-8f6a-140f7756d32d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:49.315377 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 880/1774] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:49 2022] PUT /identity/v3/projects/0ca507385d134b47aa9a588f8c5c1170/users/2e5f68ae260e44abba5d0d47647a9cd0/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:50:49.319486 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4607a09-8058-4c58-8f6a-140f7756d32d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:49.321445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a4607a09-8058-4c58-8f6a-140f7756d32d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:49.327849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 895/1775] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:50:49.341184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3073761-11a1-441a-ad92-cb2b5ae7d5d9 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.341747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3073761-11a1-441a-ad92-cb2b5ae7d5d9 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.342264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3073761-11a1-441a-ad92-cb2b5ae7d5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.369471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-f3073761-11a1-441a-ad92-cb2b5ae7d5d9 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:49.370566 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f3073761-11a1-441a-ad92-cb2b5ae7d5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:49.421695 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4607a09-8058-4c58-8f6a-140f7756d32d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:49.429027 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bfb88-e8cd-420e-8953-62547b8a9960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:49.429897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bfb88-e8cd-420e-8953-62547b8a9960 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.430345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bfb88-e8cd-420e-8953-62547b8a9960 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.430671 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bfb88-e8cd-420e-8953-62547b8a9960 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.432232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bfb88-e8cd-420e-8953-62547b8a9960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:49.434718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bfb88-e8cd-420e-8953-62547b8a9960 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:49.449707 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 881/1776] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:50:49.452061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 896/1777] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:50:49.456170 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bfb88-e8cd-420e-8953-62547b8a9960 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:49.457434 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3073761-11a1-441a-ad92-cb2b5ae7d5d9 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:49.469307 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3073761-11a1-441a-ad92-cb2b5ae7d5d9 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:49.470382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bfb88-e8cd-420e-8953-62547b8a9960 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:49.474614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e268bf-c35a-4c7c-8271-28fb835f9e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:49.475113 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e268bf-c35a-4c7c-8271-28fb835f9e6d admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.475438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e268bf-c35a-4c7c-8271-28fb835f9e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.475660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e268bf-c35a-4c7c-8271-28fb835f9e6d admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.479463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e268bf-c35a-4c7c-8271-28fb835f9e6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:50:49.479463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e268bf-c35a-4c7c-8271-28fb835f9e6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:49.479463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e268bf-c35a-4c7c-8271-28fb835f9e6d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:49.480723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60d2ff-d397-4d4d-8af0-f574e0faed40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:49.481677 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60d2ff-d397-4d4d-8af0-f574e0faed40 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.482124 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60d2ff-d397-4d4d-8af0-f574e0faed40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.482630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60d2ff-d397-4d4d-8af0-f574e0faed40 admin admin] PATH_INFO: `/v3/projects/0ca507385d134b47aa9a588f8c5c1170/users/2e5f68ae260e44abba5d0d47647a9cd0/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.486523 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60d2ff-d397-4d4d-8af0-f574e0faed40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca507385d134b47aa9a588f8c5c1170, user_id=2e5f68ae260e44abba5d0d47647a9cd0, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:49.492631 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60d2ff-d397-4d4d-8af0-f574e0faed40 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:49.493971 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 897/1778] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:50:49.494931 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e268bf-c35a-4c7c-8271-28fb835f9e6d admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:49.496695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c1e268bf-c35a-4c7c-8271-28fb835f9e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:49.510163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e268bf-c35a-4c7c-8271-28fb835f9e6d admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:49.513721 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 882/1779] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:49 2022] PUT /identity/v3/projects/0ca507385d134b47aa9a588f8c5c1170/users/2e5f68ae260e44abba5d0d47647a9cd0/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:49.514925 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60d2ff-d397-4d4d-8af0-f574e0faed40 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:49.522228 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:49.597638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60d2ff-d397-4d4d-8af0-f574e0faed40 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:49.597894 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:49.598547 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.598815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.599092 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] PATH_INFO: `/v3/projects/5f613cafcf984d5f8ca91ca26ba389f1` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.600307 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f613cafcf984d5f8ca91ca26ba389f1)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:49.602403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:49.603228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73289abd-fec7-46e7-925f-50bf8afee7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:49.603856 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73289abd-fec7-46e7-925f-50bf8afee7cd admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.604230 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73289abd-fec7-46e7-925f-50bf8afee7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.604519 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73289abd-fec7-46e7-925f-50bf8afee7cd admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.605229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73289abd-fec7-46e7-925f-50bf8afee7cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:49.607160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73289abd-fec7-46e7-925f-50bf8afee7cd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:49.635197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:49.656040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 883/1780] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:49 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:49.656424 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed2d8a-3283-47f5-8e96-2f4ffada93ea None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.656804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed2d8a-3283-47f5-8e96-2f4ffada93ea None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.657097 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed2d8a-3283-47f5-8e96-2f4ffada93ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.674925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f613cafcf984d5f8ca91ca26ba389f1', 'request_id': 'req-def1bf5e-d520-4e40-9fa1-416607a9f8e5'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:49.685768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f613cafcf984d5f8ca91ca26ba389f1', 'request_id': 'req-def1bf5e-d520-4e40-9fa1-416607a9f8e5'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:49.696272 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f613cafcf984d5f8ca91ca26ba389f1', 'request_id': 'req-def1bf5e-d520-4e40-9fa1-416607a9f8e5'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:49.699299 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-55ed2d8a-3283-47f5-8e96-2f4ffada93ea None None] MFA Rules not processed for user `2e5f68ae260e44abba5d0d47647a9cd0`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:49.701290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-55ed2d8a-3283-47f5-8e96-2f4ffada93ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:49.711697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] The token cache is being invalidate because project 5f613cafcf984d5f8ca91ca26ba389f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:50:49.712118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:49.712863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:50:49.715606 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 898/1781] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:50:49 2022] DELETE /identity/v3/projects/5f613cafcf984d5f8ca91ca26ba389f1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:49.721531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:49.725426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:49.839083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 884/1782] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:49.840817 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6421a-96ed-44e0-9b44-2fcd881a67be None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.841121 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6421a-96ed-44e0-9b44-2fcd881a67be None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.841766 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6421a-96ed-44e0-9b44-2fcd881a67be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.853969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1bf5e-d520-4e40-9fa1-416607a9f8e5 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:49.869227 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-fce6421a-96ed-44e0-9b44-2fcd881a67be None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:49.870095 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd08c6b-c3eb-4cd4-b433-b2e4e6260e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:49.871505 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd08c6b-c3eb-4cd4-b433-b2e4e6260e8f admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.871678 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fce6421a-96ed-44e0-9b44-2fcd881a67be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:49.872553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd08c6b-c3eb-4cd4-b433-b2e4e6260e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.873014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd08c6b-c3eb-4cd4-b433-b2e4e6260e8f admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.874233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd08c6b-c3eb-4cd4-b433-b2e4e6260e8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:49.878544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd08c6b-c3eb-4cd4-b433-b2e4e6260e8f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:49.890691 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 885/1783] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:49.891178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6421a-96ed-44e0-9b44-2fcd881a67be None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:49.894990 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fce6421a-96ed-44e0-9b44-2fcd881a67be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:49.916850 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 899/1784] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:49 2022] POST /identity/v3/users => generated 284 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:49.918837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018bfd4-61fe-4690-9872-55336f6198a8 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:49.919137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018bfd4-61fe-4690-9872-55336f6198a8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:49.919568 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018bfd4-61fe-4690-9872-55336f6198a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:49.961410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-f018bfd4-61fe-4690-9872-55336f6198a8 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:49.963110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f018bfd4-61fe-4690-9872-55336f6198a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:49.982386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 900/1785] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:49.982386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018bfd4-61fe-4690-9872-55336f6198a8 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:49.991487 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6421a-96ed-44e0-9b44-2fcd881a67be None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:49.998625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018bfd4-61fe-4690-9872-55336f6198a8 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:50.005078 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204d90d-a202-448c-9c43-d3885fa00462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:50.005556 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b539b2-7262-4f92-8f0e-dfa2127c0094 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:50.005733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204d90d-a202-448c-9c43-d3885fa00462 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:50.006000 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204d90d-a202-448c-9c43-d3885fa00462 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:50.006264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204d90d-a202-448c-9c43-d3885fa00462 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:50.006411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b539b2-7262-4f92-8f0e-dfa2127c0094 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:50.006669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b539b2-7262-4f92-8f0e-dfa2127c0094 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:50.006904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b539b2-7262-4f92-8f0e-dfa2127c0094 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:50.007063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204d90d-a202-448c-9c43-d3885fa00462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:50.008659 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204d90d-a202-448c-9c43-d3885fa00462 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:50.014168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b8b539b2-7262-4f92-8f0e-dfa2127c0094 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:50.025655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 901/1786] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:50.026902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204d90d-a202-448c-9c43-d3885fa00462 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:50.038397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204d90d-a202-448c-9c43-d3885fa00462 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:50.044934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2fd7b-ea89-436a-bef9-cd55f34ac350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:50.048053 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2fd7b-ea89-436a-bef9-cd55f34ac350 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:50.048677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2fd7b-ea89-436a-bef9-cd55f34ac350 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:50.049415 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2fd7b-ea89-436a-bef9-cd55f34ac350 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:50.050904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d2fd7b-ea89-436a-bef9-cd55f34ac350 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:50:50.051473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d2fd7b-ea89-436a-bef9-cd55f34ac350 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:50.054651 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d2fd7b-ea89-436a-bef9-cd55f34ac350 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:50.088630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 902/1787] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:50:50.091044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2fd7b-ea89-436a-bef9-cd55f34ac350 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:50.100632 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:50.113988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2fd7b-ea89-436a-bef9-cd55f34ac350 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:50.127134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a2bab-7024-46b9-8b94-b14e19a6ba68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:50.129854 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a2bab-7024-46b9-8b94-b14e19a6ba68 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:50.130434 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a2bab-7024-46b9-8b94-b14e19a6ba68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:50.130953 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a2bab-7024-46b9-8b94-b14e19a6ba68 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:50.132150 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a2bab-7024-46b9-8b94-b14e19a6ba68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:50.136832 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a2bab-7024-46b9-8b94-b14e19a6ba68 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:50.142160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b539b2-7262-4f92-8f0e-dfa2127c0094 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:50.143929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b539b2-7262-4f92-8f0e-dfa2127c0094 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:50.161619 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 903/1788] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:50.163176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a2bab-7024-46b9-8b94-b14e19a6ba68 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:50.163978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 886/1789] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:50.165573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b539b2-7262-4f92-8f0e-dfa2127c0094 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:50.183937 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a2bab-7024-46b9-8b94-b14e19a6ba68 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:50.186363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b539b2-7262-4f92-8f0e-dfa2127c0094 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:50.193883 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f85042-f3b1-4416-99b2-de620b0e7075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:50.194973 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85042-f3b1-4416-99b2-de620b0e7075 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:50.195479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85042-f3b1-4416-99b2-de620b0e7075 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:50.196042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f85042-f3b1-4416-99b2-de620b0e7075 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:50.197230 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f85042-f3b1-4416-99b2-de620b0e7075 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:50.201110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f85042-f3b1-4416-99b2-de620b0e7075 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:50.228670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112eeda1-082a-457d-93fb-c89dc99dd540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:50.228670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eeda1-082a-457d-93fb-c89dc99dd540 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:50.228670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eeda1-082a-457d-93fb-c89dc99dd540 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:50.228670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eeda1-082a-457d-93fb-c89dc99dd540 admin admin] PATH_INFO: `/v3/projects/c91b9bf6ef964130bf8d23778335086d/users/c0b344b009054e62906797edfb7e09df/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:50.254567 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 904/1790] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:50:50 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:50:50.261128 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f85042-f3b1-4416-99b2-de620b0e7075 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:50.262738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112eeda1-082a-457d-93fb-c89dc99dd540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c91b9bf6ef964130bf8d23778335086d, user_id=c0b344b009054e62906797edfb7e09df, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:50.268902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112eeda1-082a-457d-93fb-c89dc99dd540 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:50.288499 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 887/1791] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:50 2022] PUT /identity/v3/projects/c91b9bf6ef964130bf8d23778335086d/users/c0b344b009054e62906797edfb7e09df/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:50.313417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b081d-f6ed-4ea1-a25a-357b7e5c04ac None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:50.314131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b081d-f6ed-4ea1-a25a-357b7e5c04ac None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:50.314292 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b081d-f6ed-4ea1-a25a-357b7e5c04ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:50.355836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:50.380120 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-4a0b081d-f6ed-4ea1-a25a-357b7e5c04ac None None] MFA Rules not processed for user `c0b344b009054e62906797edfb7e09df`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:50.381479 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4a0b081d-f6ed-4ea1-a25a-357b7e5c04ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:50.430804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f85042-f3b1-4416-99b2-de620b0e7075 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:50.498371 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 888/1792] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:50:50.515684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b081d-f6ed-4ea1-a25a-357b7e5c04ac None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:50.517831 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4a0b081d-f6ed-4ea1-a25a-357b7e5c04ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:50.625767 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6059df4-832d-4d91-a896-653d82f76a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:50.627677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6059df4-832d-4d91-a896-653d82f76a05 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:50.629125 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6059df4-832d-4d91-a896-653d82f76a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:50.630105 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6059df4-832d-4d91-a896-653d82f76a05 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:50.632384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6059df4-832d-4d91-a896-653d82f76a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:50.638703 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6059df4-832d-4d91-a896-653d82f76a05 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:50.707547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b081d-f6ed-4ea1-a25a-357b7e5c04ac None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:50.715037 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582e801-76a7-4674-8af3-14e4f8c6271d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:50.716621 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582e801-76a7-4674-8af3-14e4f8c6271d service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:50.717106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582e801-76a7-4674-8af3-14e4f8c6271d service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:50.717730 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582e801-76a7-4674-8af3-14e4f8c6271d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:50.722050 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 905/1793] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:50:50 2022] POST /identity/v3/users => generated 287 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:50:50.722965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6059df4-832d-4d91-a896-653d82f76a05 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:50.737934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582e801-76a7-4674-8af3-14e4f8c6271d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:50.739976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582e801-76a7-4674-8af3-14e4f8c6271d service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:50.753722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 889/1794] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:50:50.756372 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6059df4-832d-4d91-a896-653d82f76a05 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:50.763804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b976c5-b2bb-4597-a5c2-32c35277b188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:50.765707 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b976c5-b2bb-4597-a5c2-32c35277b188 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:50.766168 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b976c5-b2bb-4597-a5c2-32c35277b188 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:50.766626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b976c5-b2bb-4597-a5c2-32c35277b188 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:50.767612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b976c5-b2bb-4597-a5c2-32c35277b188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:50.770506 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b976c5-b2bb-4597-a5c2-32c35277b188 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:50.794386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 906/1795] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:50.800232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582e801-76a7-4674-8af3-14e4f8c6271d service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:50.831753 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582e801-76a7-4674-8af3-14e4f8c6271d service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:50.842684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334b1e1-07b1-4c3d-a178-7c8bf809dda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:50.843758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334b1e1-07b1-4c3d-a178-7c8bf809dda1 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:50.844280 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334b1e1-07b1-4c3d-a178-7c8bf809dda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:50.844810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334b1e1-07b1-4c3d-a178-7c8bf809dda1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:50.845993 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334b1e1-07b1-4c3d-a178-7c8bf809dda1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:50.851317 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334b1e1-07b1-4c3d-a178-7c8bf809dda1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:50.881947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 890/1796] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:50:50.895417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b976c5-b2bb-4597-a5c2-32c35277b188 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:50.940784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b976c5-b2bb-4597-a5c2-32c35277b188 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:50.947329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e12cf-b5e9-4580-8407-8f90d1f8159c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:50.948110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e12cf-b5e9-4580-8407-8f90d1f8159c admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:50.948390 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e12cf-b5e9-4580-8407-8f90d1f8159c admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:50.948665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e12cf-b5e9-4580-8407-8f90d1f8159c admin admin] PATH_INFO: `/v3/projects/057a043641024680b2a172d719263888/users/ebdefd4854e847d9af2d0b1705b5b062/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:50.974950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e12cf-b5e9-4580-8407-8f90d1f8159c admin admin] RBAC: Authorizing `identity:create_grant(project_id=057a043641024680b2a172d719263888, user_id=ebdefd4854e847d9af2d0b1705b5b062, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:50.980099 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e12cf-b5e9-4580-8407-8f90d1f8159c admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:51.004955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 907/1797] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:50:50 2022] PUT /identity/v3/projects/057a043641024680b2a172d719263888/users/ebdefd4854e847d9af2d0b1705b5b062/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:50:51.022155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddec14d-b268-4cb0-8e6c-db35e998610d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:51.022433 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddec14d-b268-4cb0-8e6c-db35e998610d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:51.022703 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddec14d-b268-4cb0-8e6c-db35e998610d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:51.073707 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-fddec14d-b268-4cb0-8e6c-db35e998610d None None] MFA Rules not processed for user `ebdefd4854e847d9af2d0b1705b5b062`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:51.076138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fddec14d-b268-4cb0-8e6c-db35e998610d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:51.187799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 891/1798] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:50:51.205254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e12cf-b5e9-4580-8407-8f90d1f8159c admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:51.286869 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:51.358564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e12cf-b5e9-4580-8407-8f90d1f8159c admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:51.377678 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf12f59-4a03-4e1b-9310-6abf42c9fbf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:51.378831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf12f59-4a03-4e1b-9310-6abf42c9fbf1 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:51.379418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf12f59-4a03-4e1b-9310-6abf42c9fbf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:51.380041 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf12f59-4a03-4e1b-9310-6abf42c9fbf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:51.417642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa0bc4-b5f6-4a52-b3e8-9b060ec9c0d9 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:51.418198 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa0bc4-b5f6-4a52-b3e8-9b060ec9c0d9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:51.418198 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa0bc4-b5f6-4a52-b3e8-9b060ec9c0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:51.426469 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf12f59-4a03-4e1b-9310-6abf42c9fbf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:51.431549 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf12f59-4a03-4e1b-9310-6abf42c9fbf1 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:51.445014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-40fa0bc4-b5f6-4a52-b3e8-9b060ec9c0d9 None None] MFA Rules not processed for user `af1daddbc1b84ab79f269e124cbea4fb`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:50:51.446331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-40fa0bc4-b5f6-4a52-b3e8-9b060ec9c0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:51.466181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:50:51.469787 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 908/1799] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:51.564138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 892/1800] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:50:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:50:53.291582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf12f59-4a03-4e1b-9310-6abf42c9fbf1 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:53.301303 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf12f59-4a03-4e1b-9310-6abf42c9fbf1 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:53.305973 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf21ee8-15da-47e5-bba4-5b7690c78e4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:53.306730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf21ee8-15da-47e5-bba4-5b7690c78e4d service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:53.307020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf21ee8-15da-47e5-bba4-5b7690c78e4d service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:53.307438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf21ee8-15da-47e5-bba4-5b7690c78e4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:53.309741 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8bf21ee8-15da-47e5-bba4-5b7690c78e4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:50:53.389577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf21ee8-15da-47e5-bba4-5b7690c78e4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:53.391313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf21ee8-15da-47e5-bba4-5b7690c78e4d service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:53.403609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 909/1801] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:50:54.197199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa0bc4-b5f6-4a52-b3e8-9b060ec9c0d9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:50:54.211351 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa0bc4-b5f6-4a52-b3e8-9b060ec9c0d9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:50:54.218516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bddca2c-f2be-4f51-b261-e57ce2f20a11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:50:54.219572 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddca2c-f2be-4f51-b261-e57ce2f20a11 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:50:54.220077 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddca2c-f2be-4f51-b261-e57ce2f20a11 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:50:54.220519 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddca2c-f2be-4f51-b261-e57ce2f20a11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:50:54.309054 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bddca2c-f2be-4f51-b261-e57ce2f20a11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:50:54.310554 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bddca2c-f2be-4f51-b261-e57ce2f20a11 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:50:54.321987 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 893/1802] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:50:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:00.756303 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e91bca-1bcb-4ba7-9526-16bc2198b032 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:00.757315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e91bca-1bcb-4ba7-9526-16bc2198b032 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:00.757315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e91bca-1bcb-4ba7-9526-16bc2198b032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:00.787126 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-98e91bca-1bcb-4ba7-9526-16bc2198b032 None None] MFA Rules not processed for user `5150c06020594efe957359f641eba5da`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:00.790134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-98e91bca-1bcb-4ba7-9526-16bc2198b032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:00.911087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 910/1803] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:04.317380 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=46 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:09.595438 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bddca2c-f2be-4f51-b261-e57ce2f20a11 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:09.597352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6bddca2c-f2be-4f51-b261-e57ce2f20a11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:09.679141 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bddca2c-f2be-4f51-b261-e57ce2f20a11 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:09.684971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fa950-cd91-4222-86f3-f94f4bd7aa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:09.686049 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fa950-cd91-4222-86f3-f94f4bd7aa15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:09.686737 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fa950-cd91-4222-86f3-f94f4bd7aa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:09.687145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fa950-cd91-4222-86f3-f94f4bd7aa15 admin admin] PATH_INFO: `/v3/users/8081f0a1c2e14ab296052a79aa0ae7a3` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:09.688452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745fa950-cd91-4222-86f3-f94f4bd7aa15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8081f0a1c2e14ab296052a79aa0ae7a3)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:09.690918 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745fa950-cd91-4222-86f3-f94f4bd7aa15 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:09.749471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-745fa950-cd91-4222-86f3-f94f4bd7aa15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8081f0a1c2e14ab296052a79aa0ae7a3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:09.750272 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-745fa950-cd91-4222-86f3-f94f4bd7aa15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8081f0a1c2e14ab296052a79aa0ae7a3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:09.750922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-745fa950-cd91-4222-86f3-f94f4bd7aa15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8081f0a1c2e14ab296052a79aa0ae7a3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:09.766579 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-745fa950-cd91-4222-86f3-f94f4bd7aa15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8081f0a1c2e14ab296052a79aa0ae7a3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:09.779717 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-745fa950-cd91-4222-86f3-f94f4bd7aa15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8081f0a1c2e14ab296052a79aa0ae7a3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:09.802696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 894/1804] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:09 2022] DELETE /identity/v3/users/8081f0a1c2e14ab296052a79aa0ae7a3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:09.808094 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014608f8-91f7-4231-b502-e70193f29079 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:09.808895 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014608f8-91f7-4231-b502-e70193f29079 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:09.809048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014608f8-91f7-4231-b502-e70193f29079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:09.835980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-014608f8-91f7-4231-b502-e70193f29079 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:09.838014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-014608f8-91f7-4231-b502-e70193f29079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:09.937963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 911/1805] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:09.949246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fa950-cd91-4222-86f3-f94f4bd7aa15 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:09.952178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-745fa950-cd91-4222-86f3-f94f4bd7aa15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:10.046065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fa950-cd91-4222-86f3-f94f4bd7aa15 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:10.052278 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68fd0a-f4cd-49f6-b6f8-be22e5eafe54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:10.053321 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68fd0a-f4cd-49f6-b6f8-be22e5eafe54 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:10.053794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68fd0a-f4cd-49f6-b6f8-be22e5eafe54 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:10.054286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68fd0a-f4cd-49f6-b6f8-be22e5eafe54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:10.071662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68fd0a-f4cd-49f6-b6f8-be22e5eafe54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:10.074507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68fd0a-f4cd-49f6-b6f8-be22e5eafe54 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:10.095618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 895/1806] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:51:10.789396 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014608f8-91f7-4231-b502-e70193f29079 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:10.792115 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-014608f8-91f7-4231-b502-e70193f29079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:10.811154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014608f8-91f7-4231-b502-e70193f29079 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:10.824468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:10.826106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:10.826668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:10.827770 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] PATH_INFO: `/v3/projects/8c0683d58cdb43408f408d9301909caf` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:10.829860 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c0683d58cdb43408f408d9301909caf)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:10.833516 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:10.856913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:10.931779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c0683d58cdb43408f408d9301909caf', 'request_id': 'req-eb58c635-0453-42a4-a357-2382b3d8fa25'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:10.943113 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c0683d58cdb43408f408d9301909caf', 'request_id': 'req-eb58c635-0453-42a4-a357-2382b3d8fa25'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:10.954719 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c0683d58cdb43408f408d9301909caf', 'request_id': 'req-eb58c635-0453-42a4-a357-2382b3d8fa25'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:10.974908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] The token cache is being invalidate because project 8c0683d58cdb43408f408d9301909caf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:10.974908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:10.975211 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:10.978163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 912/1807] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:10 2022] DELETE /identity/v3/projects/8c0683d58cdb43408f408d9301909caf => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:10.985741 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68fd0a-f4cd-49f6-b6f8-be22e5eafe54 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:11.004023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68fd0a-f4cd-49f6-b6f8-be22e5eafe54 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:11.068318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:11.155751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc117944-8969-4e49-9bd1-14ab3afd14bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:11.157172 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc117944-8969-4e49-9bd1-14ab3afd14bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:11.157471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc117944-8969-4e49-9bd1-14ab3afd14bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:11.157888 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc117944-8969-4e49-9bd1-14ab3afd14bf admin admin] PATH_INFO: `/v3/users/8a5dcdad9c804abda02c0ba66e45d5e2` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:11.160062 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc117944-8969-4e49-9bd1-14ab3afd14bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a5dcdad9c804abda02c0ba66e45d5e2)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:11.163905 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc117944-8969-4e49-9bd1-14ab3afd14bf admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:11.238102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-cc117944-8969-4e49-9bd1-14ab3afd14bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a5dcdad9c804abda02c0ba66e45d5e2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:11.238997 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-cc117944-8969-4e49-9bd1-14ab3afd14bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a5dcdad9c804abda02c0ba66e45d5e2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:11.239761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-cc117944-8969-4e49-9bd1-14ab3afd14bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a5dcdad9c804abda02c0ba66e45d5e2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:11.254794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-cc117944-8969-4e49-9bd1-14ab3afd14bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a5dcdad9c804abda02c0ba66e45d5e2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:11.265804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-cc117944-8969-4e49-9bd1-14ab3afd14bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a5dcdad9c804abda02c0ba66e45d5e2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:11.283224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 896/1808] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:10 2022] DELETE /identity/v3/users/8a5dcdad9c804abda02c0ba66e45d5e2 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:11.717945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:11.721211 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:11.768484 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb58c635-0453-42a4-a357-2382b3d8fa25 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:11.981749 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:11.983213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:11.985085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:11.985394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] PATH_INFO: `/v3/projects/4662a487309e4da29cdb223ea2254a4d` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:11.988053 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4662a487309e4da29cdb223ea2254a4d)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:11.993159 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:12.123832 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4662a487309e4da29cdb223ea2254a4d', 'request_id': 'req-d59ce440-a709-4c47-973f-b91b4fb994d7'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.139293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4662a487309e4da29cdb223ea2254a4d', 'request_id': 'req-d59ce440-a709-4c47-973f-b91b4fb994d7'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.152498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4662a487309e4da29cdb223ea2254a4d', 'request_id': 'req-d59ce440-a709-4c47-973f-b91b4fb994d7'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.169752 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] The token cache is being invalidate because project 4662a487309e4da29cdb223ea2254a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:12.170203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.171724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.174092 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 913/1809] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:11 2022] DELETE /identity/v3/projects/4662a487309e4da29cdb223ea2254a4d => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:12.179472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc117944-8969-4e49-9bd1-14ab3afd14bf admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:12.181521 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cc117944-8969-4e49-9bd1-14ab3afd14bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:12.202854 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc117944-8969-4e49-9bd1-14ab3afd14bf admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:12.262866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:12.265195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:12.331638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:12.346147 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c396ea7-6126-42a7-81b1-90f549d265cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:12.347192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c396ea7-6126-42a7-81b1-90f549d265cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:12.347698 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c396ea7-6126-42a7-81b1-90f549d265cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:12.348197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c396ea7-6126-42a7-81b1-90f549d265cf admin admin] PATH_INFO: `/v3/users/2e5f68ae260e44abba5d0d47647a9cd0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:12.350861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c396ea7-6126-42a7-81b1-90f549d265cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e5f68ae260e44abba5d0d47647a9cd0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:12.356362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c396ea7-6126-42a7-81b1-90f549d265cf admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:12.404888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ce440-a709-4c47-973f-b91b4fb994d7 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:12.417145 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85b848-5780-42b8-929d-1069f086899e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:12.418810 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85b848-5780-42b8-929d-1069f086899e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:12.419333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85b848-5780-42b8-929d-1069f086899e admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:12.419866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85b848-5780-42b8-929d-1069f086899e admin admin] PATH_INFO: `/v3/users/c0b344b009054e62906797edfb7e09df` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:12.423006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85b848-5780-42b8-929d-1069f086899e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0b344b009054e62906797edfb7e09df)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:12.425840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85b848-5780-42b8-929d-1069f086899e admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:12.464974 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:12.473842 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3c396ea7-6126-42a7-81b1-90f549d265cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e5f68ae260e44abba5d0d47647a9cd0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.474684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3c396ea7-6126-42a7-81b1-90f549d265cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e5f68ae260e44abba5d0d47647a9cd0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.476280 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3c396ea7-6126-42a7-81b1-90f549d265cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e5f68ae260e44abba5d0d47647a9cd0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.498785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3c396ea7-6126-42a7-81b1-90f549d265cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e5f68ae260e44abba5d0d47647a9cd0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.510075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-db85b848-5780-42b8-929d-1069f086899e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0b344b009054e62906797edfb7e09df'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.516231 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3c396ea7-6126-42a7-81b1-90f549d265cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e5f68ae260e44abba5d0d47647a9cd0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.526562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-db85b848-5780-42b8-929d-1069f086899e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0b344b009054e62906797edfb7e09df'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.528060 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-db85b848-5780-42b8-929d-1069f086899e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0b344b009054e62906797edfb7e09df'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.530023 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-db85b848-5780-42b8-929d-1069f086899e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0b344b009054e62906797edfb7e09df'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.541886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-db85b848-5780-42b8-929d-1069f086899e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0b344b009054e62906797edfb7e09df'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:12.542235 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 897/1810] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:12 2022] DELETE /identity/v3/users/2e5f68ae260e44abba5d0d47647a9cd0 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:51:12.573371 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 914/1811] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:12 2022] DELETE /identity/v3/users/c0b344b009054e62906797edfb7e09df => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:51:12.580091 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ca495-b76e-4e15-adaf-d571a0fb0109 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:12.580565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ca495-b76e-4e15-adaf-d571a0fb0109 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:12.581028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ca495-b76e-4e15-adaf-d571a0fb0109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:12.610426 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-dd6ca495-b76e-4e15-adaf-d571a0fb0109 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:12.611768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dd6ca495-b76e-4e15-adaf-d571a0fb0109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:12.696998 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 898/1812] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:12.710071 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85b848-5780-42b8-929d-1069f086899e admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:12.711697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-db85b848-5780-42b8-929d-1069f086899e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:12.802978 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85b848-5780-42b8-929d-1069f086899e admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:12.809707 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e9126-dad5-4b88-b71a-2fb489a5d94d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:12.810892 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e9126-dad5-4b88-b71a-2fb489a5d94d service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:12.812778 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e9126-dad5-4b88-b71a-2fb489a5d94d service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:12.812778 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e9126-dad5-4b88-b71a-2fb489a5d94d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:12.825978 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e9126-dad5-4b88-b71a-2fb489a5d94d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:12.827701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e9126-dad5-4b88-b71a-2fb489a5d94d service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:12.844200 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 915/1813] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:13.016035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ca495-b76e-4e15-adaf-d571a0fb0109 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:13.017621 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dd6ca495-b76e-4e15-adaf-d571a0fb0109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:13.031006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ca495-b76e-4e15-adaf-d571a0fb0109 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:13.037744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147d6d8-a60b-402a-bcea-0dc35a407259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:13.038524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147d6d8-a60b-402a-bcea-0dc35a407259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:13.039489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147d6d8-a60b-402a-bcea-0dc35a407259 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:13.039785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147d6d8-a60b-402a-bcea-0dc35a407259 admin admin] PATH_INFO: `/v3/projects/0ca507385d134b47aa9a588f8c5c1170` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:13.041356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147d6d8-a60b-402a-bcea-0dc35a407259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ca507385d134b47aa9a588f8c5c1170)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:13.043742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147d6d8-a60b-402a-bcea-0dc35a407259 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:13.126423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e9126-dad5-4b88-b71a-2fb489a5d94d service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:13.130523 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b18e9126-dad5-4b88-b71a-2fb489a5d94d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:13.171114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e9126-dad5-4b88-b71a-2fb489a5d94d service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:13.172909 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3147d6d8-a60b-402a-bcea-0dc35a407259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ca507385d134b47aa9a588f8c5c1170', 'request_id': 'req-3147d6d8-a60b-402a-bcea-0dc35a407259'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.193295 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3147d6d8-a60b-402a-bcea-0dc35a407259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ca507385d134b47aa9a588f8c5c1170', 'request_id': 'req-3147d6d8-a60b-402a-bcea-0dc35a407259'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.210433 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3147d6d8-a60b-402a-bcea-0dc35a407259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ca507385d134b47aa9a588f8c5c1170', 'request_id': 'req-3147d6d8-a60b-402a-bcea-0dc35a407259'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.226639 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3147d6d8-a60b-402a-bcea-0dc35a407259 admin admin] The token cache is being invalidate because project 0ca507385d134b47aa9a588f8c5c1170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:13.227391 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3147d6d8-a60b-402a-bcea-0dc35a407259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.229183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3147d6d8-a60b-402a-bcea-0dc35a407259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.231710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 899/1814] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:13 2022] DELETE /identity/v3/projects/0ca507385d134b47aa9a588f8c5c1170 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:51:13.237152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050ddd-aab1-41e9-af36-95b96c8814bb None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:13.237583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050ddd-aab1-41e9-af36-95b96c8814bb None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:13.237992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a050ddd-aab1-41e9-af36-95b96c8814bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:13.262861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6a050ddd-aab1-41e9-af36-95b96c8814bb None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:13.264446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6a050ddd-aab1-41e9-af36-95b96c8814bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:13.312060 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:13.329124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede9f40-2f4e-4b9c-bc5e-b0fbc97a84d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:13.329536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede9f40-2f4e-4b9c-bc5e-b0fbc97a84d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:13.329536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede9f40-2f4e-4b9c-bc5e-b0fbc97a84d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:13.330213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede9f40-2f4e-4b9c-bc5e-b0fbc97a84d4 admin admin] PATH_INFO: `/v3/users/ebdefd4854e847d9af2d0b1705b5b062` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:13.331699 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede9f40-2f4e-4b9c-bc5e-b0fbc97a84d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebdefd4854e847d9af2d0b1705b5b062)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:13.334082 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede9f40-2f4e-4b9c-bc5e-b0fbc97a84d4 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:13.341239 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:13.385566 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 900/1815] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:13.396423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d54ce-2f7c-42a7-a154-513824951cad None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:13.397309 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d54ce-2f7c-42a7-a154-513824951cad None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:13.398037 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d54ce-2f7c-42a7-a154-513824951cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:13.424076 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-8ede9f40-2f4e-4b9c-bc5e-b0fbc97a84d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebdefd4854e847d9af2d0b1705b5b062'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.434201 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-8ede9f40-2f4e-4b9c-bc5e-b0fbc97a84d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebdefd4854e847d9af2d0b1705b5b062'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.435069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-8ede9f40-2f4e-4b9c-bc5e-b0fbc97a84d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebdefd4854e847d9af2d0b1705b5b062'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.435681 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-8ede9f40-2f4e-4b9c-bc5e-b0fbc97a84d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebdefd4854e847d9af2d0b1705b5b062'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.448940 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-fd3d54ce-2f7c-42a7-a154-513824951cad None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:13.450575 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-8ede9f40-2f4e-4b9c-bc5e-b0fbc97a84d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebdefd4854e847d9af2d0b1705b5b062'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.451076 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fd3d54ce-2f7c-42a7-a154-513824951cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:13.468199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 901/1816] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:13.476429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d54ce-2f7c-42a7-a154-513824951cad None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:13.501204 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d54ce-2f7c-42a7-a154-513824951cad None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:13.509231 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfac0e-c51e-41f2-a6f9-dc1cb3f6099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:13.509870 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfac0e-c51e-41f2-a6f9-dc1cb3f6099c admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:13.510152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfac0e-c51e-41f2-a6f9-dc1cb3f6099c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:13.510387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfac0e-c51e-41f2-a6f9-dc1cb3f6099c admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:13.511102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfac0e-c51e-41f2-a6f9-dc1cb3f6099c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:51:13.511381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfac0e-c51e-41f2-a6f9-dc1cb3f6099c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:13.513522 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfac0e-c51e-41f2-a6f9-dc1cb3f6099c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:13.513868 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 916/1817] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:13 2022] DELETE /identity/v3/users/ebdefd4854e847d9af2d0b1705b5b062 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:13.515809 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede9f40-2f4e-4b9c-bc5e-b0fbc97a84d4 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:13.519056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8ede9f40-2f4e-4b9c-bc5e-b0fbc97a84d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:13.550409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 902/1818] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:51:13.550409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3c56b-8aba-4ea7-b8da-c2076d2b4086 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:13.550409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3c56b-8aba-4ea7-b8da-c2076d2b4086 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:13.550409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3c56b-8aba-4ea7-b8da-c2076d2b4086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:13.570862 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d6c3c56b-8aba-4ea7-b8da-c2076d2b4086 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:13.572361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d6c3c56b-8aba-4ea7-b8da-c2076d2b4086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:13.657305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 903/1819] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:13.659016 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3c56b-8aba-4ea7-b8da-c2076d2b4086 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:13.676674 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3c56b-8aba-4ea7-b8da-c2076d2b4086 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:13.681915 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bea2aa-5547-475f-9627-971d010005ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:13.682555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bea2aa-5547-475f-9627-971d010005ff admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:13.682833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bea2aa-5547-475f-9627-971d010005ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:13.683106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bea2aa-5547-475f-9627-971d010005ff admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:13.683763 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bea2aa-5547-475f-9627-971d010005ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:13.685813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bea2aa-5547-475f-9627-971d010005ff admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:13.687837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede9f40-2f4e-4b9c-bc5e-b0fbc97a84d4 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:13.695784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:13.697056 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:13.697545 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:13.697906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] PATH_INFO: `/v3/projects/c91b9bf6ef964130bf8d23778335086d` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:13.700980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c91b9bf6ef964130bf8d23778335086d)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:13.704952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:13.726986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 904/1820] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:51:13 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:13.730184 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bea2aa-5547-475f-9627-971d010005ff admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:13.733473 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e9bea2aa-5547-475f-9627-971d010005ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:13.823118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c91b9bf6ef964130bf8d23778335086d', 'request_id': 'req-9391e549-edd3-4e61-af1b-308aa04046f9'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.846743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bea2aa-5547-475f-9627-971d010005ff admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:13.861348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c91b9bf6ef964130bf8d23778335086d', 'request_id': 'req-9391e549-edd3-4e61-af1b-308aa04046f9'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.883389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c91b9bf6ef964130bf8d23778335086d', 'request_id': 'req-9391e549-edd3-4e61-af1b-308aa04046f9'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.908673 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] The token cache is being invalidate because project c91b9bf6ef964130bf8d23778335086d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:13.909284 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.910712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:13.913627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 917/1821] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:13 2022] DELETE /identity/v3/projects/c91b9bf6ef964130bf8d23778335086d => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:13.915229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:13.917907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:13.927540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:14.019830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed0af6-dce0-4675-806d-d752ca791b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:14.021023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed0af6-dce0-4675-806d-d752ca791b9d service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:14.021482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed0af6-dce0-4675-806d-d752ca791b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:14.022457 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed0af6-dce0-4675-806d-d752ca791b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:14.026502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f5ed0af6-dce0-4675-806d-d752ca791b9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:14.048330 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:14.068866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391e549-edd3-4e61-af1b-308aa04046f9 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:14.081274 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167623c5-d21e-4265-adf3-0810e8c94f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:14.082512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167623c5-d21e-4265-adf3-0810e8c94f92 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:14.083077 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167623c5-d21e-4265-adf3-0810e8c94f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:14.083637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167623c5-d21e-4265-adf3-0810e8c94f92 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:14.085313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167623c5-d21e-4265-adf3-0810e8c94f92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:14.092457 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167623c5-d21e-4265-adf3-0810e8c94f92 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:14.155240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 918/1822] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:51:13 2022] POST /identity/v3/users => generated 292 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:14.155240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef795d-fd20-47ea-998c-7aa14499ffbc None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:14.156155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef795d-fd20-47ea-998c-7aa14499ffbc None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:14.156155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef795d-fd20-47ea-998c-7aa14499ffbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:14.184888 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed0af6-dce0-4675-806d-d752ca791b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:14.203747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-6fef795d-fd20-47ea-998c-7aa14499ffbc None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:14.207270 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6fef795d-fd20-47ea-998c-7aa14499ffbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:14.226016 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed0af6-dce0-4675-806d-d752ca791b9d service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:14.229489 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 919/1823] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:14.230402 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef795d-fd20-47ea-998c-7aa14499ffbc None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:14.241196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef795d-fd20-47ea-998c-7aa14499ffbc None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:14.247124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a155b-088f-4b29-a9bf-738cdd92f0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:14.247905 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a155b-088f-4b29-a9bf-738cdd92f0cc admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:14.248459 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a155b-088f-4b29-a9bf-738cdd92f0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:14.248667 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a155b-088f-4b29-a9bf-738cdd92f0cc admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:14.249733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a155b-088f-4b29-a9bf-738cdd92f0cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:14.251737 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a155b-088f-4b29-a9bf-738cdd92f0cc admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:14.264544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 920/1824] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:14.269444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd424f8f-862b-4784-b58f-71dffb2005e6 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:14.269809 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd424f8f-862b-4784-b58f-71dffb2005e6 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:14.270155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd424f8f-862b-4784-b58f-71dffb2005e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:14.281942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 905/1825] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:14.293560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed0af6-dce0-4675-806d-d752ca791b9d service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:14.313392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-cd424f8f-862b-4784-b58f-71dffb2005e6 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:14.317435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-cd424f8f-862b-4784-b58f-71dffb2005e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:14.331641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 921/1826] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:14.339809 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd424f8f-862b-4784-b58f-71dffb2005e6 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:14.349422 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed0af6-dce0-4675-806d-d752ca791b9d service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:14.358863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e32948-6af6-4367-a38c-d29274f5ca92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:14.359776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e32948-6af6-4367-a38c-d29274f5ca92 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:14.361079 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e32948-6af6-4367-a38c-d29274f5ca92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:14.361079 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e32948-6af6-4367-a38c-d29274f5ca92 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:14.362187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e32948-6af6-4367-a38c-d29274f5ca92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:14.364461 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e32948-6af6-4367-a38c-d29274f5ca92 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:14.369221 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd424f8f-862b-4784-b58f-71dffb2005e6 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:14.384818 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 906/1827] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:14.386616 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e32948-6af6-4367-a38c-d29274f5ca92 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:14.393025 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7372de-c894-4549-971a-1de7d9becef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:14.394284 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7372de-c894-4549-971a-1de7d9becef9 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:14.394886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7372de-c894-4549-971a-1de7d9becef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:14.395376 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7372de-c894-4549-971a-1de7d9becef9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:14.396749 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7372de-c894-4549-971a-1de7d9becef9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:51:14.397291 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7372de-c894-4549-971a-1de7d9becef9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:14.400533 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e32948-6af6-4367-a38c-d29274f5ca92 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:14.401129 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7372de-c894-4549-971a-1de7d9becef9 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:14.407449 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a99b20-2c6c-476f-8705-fe0addaebddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:14.408571 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a99b20-2c6c-476f-8705-fe0addaebddd admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:14.409068 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a99b20-2c6c-476f-8705-fe0addaebddd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:14.409542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a99b20-2c6c-476f-8705-fe0addaebddd admin admin] PATH_INFO: `/v3/projects/eb9120906e6140c7bbc978eb051e887f/users/2c58f05b4afb4181a3ec0abda7418752/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:14.459622 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a99b20-2c6c-476f-8705-fe0addaebddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9120906e6140c7bbc978eb051e887f, user_id=2c58f05b4afb4181a3ec0abda7418752, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:14.466038 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a99b20-2c6c-476f-8705-fe0addaebddd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:14.468793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 922/1828] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:51:14.473578 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7372de-c894-4549-971a-1de7d9becef9 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:14.503368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7372de-c894-4549-971a-1de7d9becef9 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:14.512628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 907/1829] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:14 2022] PUT /identity/v3/projects/eb9120906e6140c7bbc978eb051e887f/users/2c58f05b4afb4181a3ec0abda7418752/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:14.527783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd59b14-7463-410a-b8f2-0318ebcd870b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:14.529226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd59b14-7463-410a-b8f2-0318ebcd870b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:14.529697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd59b14-7463-410a-b8f2-0318ebcd870b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:14.621755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6fd59b14-7463-410a-b8f2-0318ebcd870b None None] MFA Rules not processed for user `2c58f05b4afb4181a3ec0abda7418752`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:14.625288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6fd59b14-7463-410a-b8f2-0318ebcd870b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:14.682884 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609d5c5-569e-45f8-ba55-5a0e6824dbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:14.684039 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609d5c5-569e-45f8-ba55-5a0e6824dbd0 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:14.684504 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609d5c5-569e-45f8-ba55-5a0e6824dbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:14.684971 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609d5c5-569e-45f8-ba55-5a0e6824dbd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:14.686426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609d5c5-569e-45f8-ba55-5a0e6824dbd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:14.692627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609d5c5-569e-45f8-ba55-5a0e6824dbd0 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:14.766969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 923/1830] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:51:14 2022] POST /identity/v3/projects => generated 365 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:14.769049 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609d5c5-569e-45f8-ba55-5a0e6824dbd0 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:14.942416 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:14.977174 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 908/1831] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:14.983680 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609d5c5-569e-45f8-ba55-5a0e6824dbd0 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:14.996537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebc74b-8f4c-43e6-835b-de02579d947b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:14.997238 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebc74b-8f4c-43e6-835b-de02579d947b admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:14.997612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebc74b-8f4c-43e6-835b-de02579d947b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:14.997979 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebc74b-8f4c-43e6-835b-de02579d947b admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:14.998883 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebc74b-8f4c-43e6-835b-de02579d947b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:15.001752 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebc74b-8f4c-43e6-835b-de02579d947b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:15.010829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd59b14-7463-410a-b8f2-0318ebcd870b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:15.013583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6fd59b14-7463-410a-b8f2-0318ebcd870b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:15.071121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 924/1832] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:51:14 2022] POST /identity/v3/users => generated 283 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:51:15.073677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebc74b-8f4c-43e6-835b-de02579d947b admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:15.101343 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebc74b-8f4c-43e6-835b-de02579d947b admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:15.128508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df2d0f-0b1f-43eb-88ef-844777dd25a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:15.132409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df2d0f-0b1f-43eb-88ef-844777dd25a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:15.134173 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df2d0f-0b1f-43eb-88ef-844777dd25a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:15.136096 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df2d0f-0b1f-43eb-88ef-844777dd25a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:15.137745 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df2d0f-0b1f-43eb-88ef-844777dd25a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:15.141474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df2d0f-0b1f-43eb-88ef-844777dd25a8 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:15.177729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 925/1833] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:15.183949 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df2d0f-0b1f-43eb-88ef-844777dd25a8 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:15.193146 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd59b14-7463-410a-b8f2-0318ebcd870b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:15.215769 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df2d0f-0b1f-43eb-88ef-844777dd25a8 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:15.237994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:15.239358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f813a1-7ede-434d-91f5-f46a90602da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:15.240864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f813a1-7ede-434d-91f5-f46a90602da4 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:15.241674 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f813a1-7ede-434d-91f5-f46a90602da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:15.242621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f813a1-7ede-434d-91f5-f46a90602da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:15.244535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f813a1-7ede-434d-91f5-f46a90602da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:15.248088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f813a1-7ede-434d-91f5-f46a90602da4 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:15.300767 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 926/1834] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:15.306635 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f813a1-7ede-434d-91f5-f46a90602da4 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:15.347011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f813a1-7ede-434d-91f5-f46a90602da4 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:15.358084 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632153dd-7c07-47bb-835e-6f855c2eb70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:15.360286 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632153dd-7c07-47bb-835e-6f855c2eb70f admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:15.360866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632153dd-7c07-47bb-835e-6f855c2eb70f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:15.361536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632153dd-7c07-47bb-835e-6f855c2eb70f admin admin] PATH_INFO: `/v3/projects/ab033e1c6d8146d4a1a873a4d40393a9/users/a548f47db56c4a19abb9bffeba45fd02/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:15.396518 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632153dd-7c07-47bb-835e-6f855c2eb70f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab033e1c6d8146d4a1a873a4d40393a9, user_id=a548f47db56c4a19abb9bffeba45fd02, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:15.398315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69698cd-0e2a-4313-9fdf-fb3cc17411dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:15.398999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69698cd-0e2a-4313-9fdf-fb3cc17411dc service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:15.399273 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69698cd-0e2a-4313-9fdf-fb3cc17411dc service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:15.399562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69698cd-0e2a-4313-9fdf-fb3cc17411dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:15.402549 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632153dd-7c07-47bb-835e-6f855c2eb70f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:15.463176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 927/1835] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:15 2022] PUT /identity/v3/projects/ab033e1c6d8146d4a1a873a4d40393a9/users/a548f47db56c4a19abb9bffeba45fd02/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:15.463176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632153dd-7c07-47bb-835e-6f855c2eb70f admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:15.463176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-632153dd-7c07-47bb-835e-6f855c2eb70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:15.509288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69698cd-0e2a-4313-9fdf-fb3cc17411dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:15.512825 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69698cd-0e2a-4313-9fdf-fb3cc17411dc service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:15.531921 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 909/1836] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:15.534975 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8981e35-7035-49de-b20b-7d8bac4b0afb None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:15.534975 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8981e35-7035-49de-b20b-7d8bac4b0afb None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:15.535359 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8981e35-7035-49de-b20b-7d8bac4b0afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:15.569935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632153dd-7c07-47bb-835e-6f855c2eb70f admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:15.577386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:15.578406 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:15.578813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:15.579164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684 admin admin] PATH_INFO: `/v3/projects/057a043641024680b2a172d719263888` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:15.580830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684 admin admin] RBAC: Authorizing `identity:delete_project(project_id=057a043641024680b2a172d719263888)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:15.583756 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:15.600942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f8981e35-7035-49de-b20b-7d8bac4b0afb None None] MFA Rules not processed for user `a548f47db56c4a19abb9bffeba45fd02`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:15.603518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f8981e35-7035-49de-b20b-7d8bac4b0afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:15.681069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '057a043641024680b2a172d719263888', 'request_id': 'req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:15.692766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '057a043641024680b2a172d719263888', 'request_id': 'req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:15.702595 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '057a043641024680b2a172d719263888', 'request_id': 'req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:15.715130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684 admin admin] The token cache is being invalidate because project 057a043641024680b2a172d719263888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:15.715530 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:15.716417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9931a33a-7eb2-4a41-bdf5-4ad3e583f684 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:15.718498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 928/1837] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:15 2022] DELETE /identity/v3/projects/057a043641024680b2a172d719263888 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:15.723919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79a108-15f7-408e-8d54-e1b3b5279a2d None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:15.724226 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79a108-15f7-408e-8d54-e1b3b5279a2d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:15.724737 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79a108-15f7-408e-8d54-e1b3b5279a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:15.747997 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 910/1838] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:15.748510 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-4e79a108-15f7-408e-8d54-e1b3b5279a2d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:15.749800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4e79a108-15f7-408e-8d54-e1b3b5279a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:15.753109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8981e35-7035-49de-b20b-7d8bac4b0afb None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:15.754875 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f8981e35-7035-49de-b20b-7d8bac4b0afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:15.762952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:15.779047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8981e35-7035-49de-b20b-7d8bac4b0afb None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:15.879049 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 929/1839] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:51:15.883516 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50182d45-0050-412a-88e6-6b268c5fcc91 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:15.884374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50182d45-0050-412a-88e6-6b268c5fcc91 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:15.885042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50182d45-0050-412a-88e6-6b268c5fcc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:15.885660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730cf68-bed9-4d5b-af80-696663ab5ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:15.886479 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730cf68-bed9-4d5b-af80-696663ab5ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:15.886754 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730cf68-bed9-4d5b-af80-696663ab5ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:15.887099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730cf68-bed9-4d5b-af80-696663ab5ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:15.888036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730cf68-bed9-4d5b-af80-696663ab5ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:15.890886 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730cf68-bed9-4d5b-af80-696663ab5ea9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:15.914501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:15.922710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-50182d45-0050-412a-88e6-6b268c5fcc91 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:15.924121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-50182d45-0050-412a-88e6-6b268c5fcc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:15.934046 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 930/1840] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:15.940041 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50182d45-0050-412a-88e6-6b268c5fcc91 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:15.949705 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 911/1841] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:51:15 2022] POST /identity/v3/projects => generated 363 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:15.956123 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730cf68-bed9-4d5b-af80-696663ab5ea9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:15.960684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50182d45-0050-412a-88e6-6b268c5fcc91 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:16.096255 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730cf68-bed9-4d5b-af80-696663ab5ea9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:16.109545 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b76683-381e-424e-b187-99a75b3cda55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:16.109904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a3d24-8162-48db-9202-67e98e04bcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:16.110099 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b76683-381e-424e-b187-99a75b3cda55 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:16.110099 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b76683-381e-424e-b187-99a75b3cda55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:16.110392 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a3d24-8162-48db-9202-67e98e04bcbf admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:16.110573 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b76683-381e-424e-b187-99a75b3cda55 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:16.110732 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a3d24-8162-48db-9202-67e98e04bcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:16.110732 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a3d24-8162-48db-9202-67e98e04bcbf admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:16.111037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b76683-381e-424e-b187-99a75b3cda55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:51:16.111037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b76683-381e-424e-b187-99a75b3cda55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:16.111318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a3d24-8162-48db-9202-67e98e04bcbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:16.113726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b76683-381e-424e-b187-99a75b3cda55 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:16.114609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a3d24-8162-48db-9202-67e98e04bcbf admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:16.150842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 931/1842] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:51:16.152990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b76683-381e-424e-b187-99a75b3cda55 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:16.163949 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 912/1843] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:51:15 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:16.169041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a3d24-8162-48db-9202-67e98e04bcbf admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:16.187721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b76683-381e-424e-b187-99a75b3cda55 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:16.194920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a3d24-8162-48db-9202-67e98e04bcbf admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:16.208817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed96207-9caa-4e9f-a6c6-4943c75ebe43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:16.210004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed96207-9caa-4e9f-a6c6-4943c75ebe43 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:16.210599 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed96207-9caa-4e9f-a6c6-4943c75ebe43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:16.211128 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed96207-9caa-4e9f-a6c6-4943c75ebe43 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:16.212102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11cd05-43fc-4c35-a325-f3a394b4ee86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:16.212677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed96207-9caa-4e9f-a6c6-4943c75ebe43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:16.212851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11cd05-43fc-4c35-a325-f3a394b4ee86 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:16.213128 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11cd05-43fc-4c35-a325-f3a394b4ee86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:16.213396 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11cd05-43fc-4c35-a325-f3a394b4ee86 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:16.214156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11cd05-43fc-4c35-a325-f3a394b4ee86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:16.215864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11cd05-43fc-4c35-a325-f3a394b4ee86 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:16.218220 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed96207-9caa-4e9f-a6c6-4943c75ebe43 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:16.244587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 913/1844] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:51:16.252172 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11cd05-43fc-4c35-a325-f3a394b4ee86 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:16.281119 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11cd05-43fc-4c35-a325-f3a394b4ee86 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:16.283272 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 932/1845] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:51:16 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:16.290677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed96207-9caa-4e9f-a6c6-4943c75ebe43 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:16.365859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef82f78-3ad7-47f7-acc9-97241dfc8490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:16.366587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef82f78-3ad7-47f7-acc9-97241dfc8490 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:16.366867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef82f78-3ad7-47f7-acc9-97241dfc8490 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:16.367903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef82f78-3ad7-47f7-acc9-97241dfc8490 admin admin] PATH_INFO: `/v3/projects/5a33641d25bb4c18bec2f60393b571b1/users/396a88dc07cc47c9920699794601e5c6/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:16.393263 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef82f78-3ad7-47f7-acc9-97241dfc8490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a33641d25bb4c18bec2f60393b571b1, user_id=396a88dc07cc47c9920699794601e5c6, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:16.396212 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef82f78-3ad7-47f7-acc9-97241dfc8490 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:16.418818 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 914/1846] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:16 2022] PUT /identity/v3/projects/5a33641d25bb4c18bec2f60393b571b1/users/396a88dc07cc47c9920699794601e5c6/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:16.420039 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef82f78-3ad7-47f7-acc9-97241dfc8490 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:16.440448 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed96207-9caa-4e9f-a6c6-4943c75ebe43 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:16.511990 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:16.524461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:16.525808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef82f78-3ad7-47f7-acc9-97241dfc8490 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:16.533214 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd39a14-65a9-4ead-99cb-a367d8ae61eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:16.534068 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd39a14-65a9-4ead-99cb-a367d8ae61eb admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:16.534421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd39a14-65a9-4ead-99cb-a367d8ae61eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:16.534797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd39a14-65a9-4ead-99cb-a367d8ae61eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:16.535542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd39a14-65a9-4ead-99cb-a367d8ae61eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:16.537411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd39a14-65a9-4ead-99cb-a367d8ae61eb admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:16.552322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 915/1847] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:16.557429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd39a14-65a9-4ead-99cb-a367d8ae61eb admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:16.575854 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd39a14-65a9-4ead-99cb-a367d8ae61eb admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:16.582311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bc9ee-5a07-4307-979d-f51e6e9ce4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:16.583580 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bc9ee-5a07-4307-979d-f51e6e9ce4f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:16.583580 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bc9ee-5a07-4307-979d-f51e6e9ce4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:16.583840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bc9ee-5a07-4307-979d-f51e6e9ce4f3 admin admin] PATH_INFO: `/v3/projects/5a33641d25bb4c18bec2f60393b571b1/users/396a88dc07cc47c9920699794601e5c6/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:16.586608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bc9ee-5a07-4307-979d-f51e6e9ce4f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a33641d25bb4c18bec2f60393b571b1, user_id=396a88dc07cc47c9920699794601e5c6, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:16.590836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bc9ee-5a07-4307-979d-f51e6e9ce4f3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:16.603628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784995a3-c5e4-435a-9d0e-62343db1a21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:16.604749 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784995a3-c5e4-435a-9d0e-62343db1a21d admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:16.605243 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784995a3-c5e4-435a-9d0e-62343db1a21d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:16.605774 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784995a3-c5e4-435a-9d0e-62343db1a21d admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:16.606988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784995a3-c5e4-435a-9d0e-62343db1a21d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:16.608069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 916/1848] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:16 2022] PUT /identity/v3/projects/5a33641d25bb4c18bec2f60393b571b1/users/396a88dc07cc47c9920699794601e5c6/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:51:16.610898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784995a3-c5e4-435a-9d0e-62343db1a21d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:16.621929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62106b60-30b8-425b-bf56-5497529d1e4b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:16.622213 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62106b60-30b8-425b-bf56-5497529d1e4b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:16.622485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62106b60-30b8-425b-bf56-5497529d1e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:16.645823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 933/1849] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:51:16 2022] POST /identity/v3/users => generated 287 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:51:16.651569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784995a3-c5e4-435a-9d0e-62343db1a21d admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:16.662880 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784995a3-c5e4-435a-9d0e-62343db1a21d admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:16.672968 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-62106b60-30b8-425b-bf56-5497529d1e4b None None] MFA Rules not processed for user `396a88dc07cc47c9920699794601e5c6`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:16.675051 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-62106b60-30b8-425b-bf56-5497529d1e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:16.752094 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab490170-ba49-429c-944f-a12b25ef0b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:16.753176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab490170-ba49-429c-944f-a12b25ef0b74 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:16.753675 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab490170-ba49-429c-944f-a12b25ef0b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:16.754157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab490170-ba49-429c-944f-a12b25ef0b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:16.755347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab490170-ba49-429c-944f-a12b25ef0b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:16.758463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab490170-ba49-429c-944f-a12b25ef0b74 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:16.778918 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 917/1850] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:51:16.779400 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 934/1851] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:51:16.784053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62106b60-30b8-425b-bf56-5497529d1e4b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:16.792134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab490170-ba49-429c-944f-a12b25ef0b74 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:16.794773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ab490170-ba49-429c-944f-a12b25ef0b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:16.800972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62106b60-30b8-425b-bf56-5497529d1e4b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:16.808750 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c9bc5-ba4f-4d4a-8d75-ccea52dd71a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:16.809932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c9bc5-ba4f-4d4a-8d75-ccea52dd71a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:16.810447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c9bc5-ba4f-4d4a-8d75-ccea52dd71a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:16.810870 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c9bc5-ba4f-4d4a-8d75-ccea52dd71a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:16.813151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c9bc5-ba4f-4d4a-8d75-ccea52dd71a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:16.816364 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c9bc5-ba4f-4d4a-8d75-ccea52dd71a6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:16.846220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 918/1852] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:16.848124 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c9bc5-ba4f-4d4a-8d75-ccea52dd71a6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:16.866226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c9bc5-ba4f-4d4a-8d75-ccea52dd71a6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:16.875385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f104be8-53da-464d-a8e1-2345fcce0bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:16.876594 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f104be8-53da-464d-a8e1-2345fcce0bed admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:16.877093 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f104be8-53da-464d-a8e1-2345fcce0bed admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:16.877586 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f104be8-53da-464d-a8e1-2345fcce0bed admin admin] PATH_INFO: `/v3/projects/9a6be8e43c7742be84f4dc3c95a990e5/users/ed6e530c3f544748bd3ffc256b1630b6/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:16.895936 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab490170-ba49-429c-944f-a12b25ef0b74 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:16.901094 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3fee7-17ca-4ee6-8975-648a350473b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:16.902830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3fee7-17ca-4ee6-8975-648a350473b0 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:16.903090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3fee7-17ca-4ee6-8975-648a350473b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:16.903360 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3fee7-17ca-4ee6-8975-648a350473b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:16.910508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f104be8-53da-464d-a8e1-2345fcce0bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a6be8e43c7742be84f4dc3c95a990e5, user_id=ed6e530c3f544748bd3ffc256b1630b6, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:16.914775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3fee7-17ca-4ee6-8975-648a350473b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:16.915886 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f104be8-53da-464d-a8e1-2345fcce0bed admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:16.916302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3fee7-17ca-4ee6-8975-648a350473b0 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:16.928328 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 935/1853] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:51:16.937995 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 919/1854] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:16 2022] PUT /identity/v3/projects/9a6be8e43c7742be84f4dc3c95a990e5/users/ed6e530c3f544748bd3ffc256b1630b6/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:16.944130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d2e23-0c5d-47f4-8255-f8b1a7c52827 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:16.944565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d2e23-0c5d-47f4-8255-f8b1a7c52827 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:16.944565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d2e23-0c5d-47f4-8255-f8b1a7c52827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:16.966318 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-f77d2e23-0c5d-47f4-8255-f8b1a7c52827 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:16.967581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f77d2e23-0c5d-47f4-8255-f8b1a7c52827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:17.036066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:17.042585 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f104be8-53da-464d-a8e1-2345fcce0bed admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:17.053871 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 936/1855] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:17.067422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d2e23-0c5d-47f4-8255-f8b1a7c52827 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:17.069107 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f77d2e23-0c5d-47f4-8255-f8b1a7c52827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:17.118878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f104be8-53da-464d-a8e1-2345fcce0bed admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:17.125480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcf107-ba71-47f9-b6b8-5747ef308c7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:17.126199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcf107-ba71-47f9-b6b8-5747ef308c7e service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:17.126532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcf107-ba71-47f9-b6b8-5747ef308c7e service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:17.126859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcf107-ba71-47f9-b6b8-5747ef308c7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:17.168039 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d2e23-0c5d-47f4-8255-f8b1a7c52827 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:17.171956 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca8c03-1e08-4fe8-a700-2bb7b0a82581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:17.172609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca8c03-1e08-4fe8-a700-2bb7b0a82581 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:17.172884 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca8c03-1e08-4fe8-a700-2bb7b0a82581 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:17.173156 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca8c03-1e08-4fe8-a700-2bb7b0a82581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:17.182630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:17.184863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca8c03-1e08-4fe8-a700-2bb7b0a82581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:17.186364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca8c03-1e08-4fe8-a700-2bb7b0a82581 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:17.198434 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 937/1856] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:51:17.221204 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bcf107-ba71-47f9-b6b8-5747ef308c7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:17.224110 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bcf107-ba71-47f9-b6b8-5747ef308c7e service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:17.252163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 920/1857] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:51:21.850813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca8c03-1e08-4fe8-a700-2bb7b0a82581 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:21.884031 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca8c03-1e08-4fe8-a700-2bb7b0a82581 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:21.905171 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed3a85-d435-4c5c-a61a-6b80225b0961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:21.907413 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed3a85-d435-4c5c-a61a-6b80225b0961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:21.908387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed3a85-d435-4c5c-a61a-6b80225b0961 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:21.909193 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed3a85-d435-4c5c-a61a-6b80225b0961 admin admin] PATH_INFO: `/v3/users/a548f47db56c4a19abb9bffeba45fd02` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:21.919450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed3a85-d435-4c5c-a61a-6b80225b0961 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a548f47db56c4a19abb9bffeba45fd02)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:21.924072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed3a85-d435-4c5c-a61a-6b80225b0961 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:22.068042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9aed3a85-d435-4c5c-a61a-6b80225b0961 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a548f47db56c4a19abb9bffeba45fd02'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:22.120541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9aed3a85-d435-4c5c-a61a-6b80225b0961 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a548f47db56c4a19abb9bffeba45fd02'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:22.121750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9aed3a85-d435-4c5c-a61a-6b80225b0961 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a548f47db56c4a19abb9bffeba45fd02'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:22.123608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9aed3a85-d435-4c5c-a61a-6b80225b0961 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a548f47db56c4a19abb9bffeba45fd02'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:22.141914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9aed3a85-d435-4c5c-a61a-6b80225b0961 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a548f47db56c4a19abb9bffeba45fd02'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:22.183271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 938/1858] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:21 2022] DELETE /identity/v3/users/a548f47db56c4a19abb9bffeba45fd02 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:22.189105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c18dc-ccfd-4e82-831e-710ffa098253 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:22.189931 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c18dc-ccfd-4e82-831e-710ffa098253 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:22.190070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c18dc-ccfd-4e82-831e-710ffa098253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:22.229960 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5f9c18dc-ccfd-4e82-831e-710ffa098253 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:22.233190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5f9c18dc-ccfd-4e82-831e-710ffa098253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:22.305927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed3a85-d435-4c5c-a61a-6b80225b0961 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:22.308745 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9aed3a85-d435-4c5c-a61a-6b80225b0961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:22.450340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 921/1859] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:51:22.462248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c18dc-ccfd-4e82-831e-710ffa098253 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:22.464917 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5f9c18dc-ccfd-4e82-831e-710ffa098253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:22.587363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed3a85-d435-4c5c-a61a-6b80225b0961 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:22.618856 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6f86f-caab-4e32-9e96-8f524182dd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:22.625311 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6f86f-caab-4e32-9e96-8f524182dd50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:22.625793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6f86f-caab-4e32-9e96-8f524182dd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:22.625980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6f86f-caab-4e32-9e96-8f524182dd50 admin admin] PATH_INFO: `/v3/users/2c58f05b4afb4181a3ec0abda7418752` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:22.640084 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6f86f-caab-4e32-9e96-8f524182dd50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c58f05b4afb4181a3ec0abda7418752)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:22.647040 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6f86f-caab-4e32-9e96-8f524182dd50 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:22.702271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:22.703804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c18dc-ccfd-4e82-831e-710ffa098253 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:22.722118 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27851cc9-5955-4a09-993f-d7b82eb5be22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:22.722781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27851cc9-5955-4a09-993f-d7b82eb5be22 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:22.723075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27851cc9-5955-4a09-993f-d7b82eb5be22 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:22.723403 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27851cc9-5955-4a09-993f-d7b82eb5be22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:22.759481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27851cc9-5955-4a09-993f-d7b82eb5be22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:22.770117 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27851cc9-5955-4a09-993f-d7b82eb5be22 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:22.843625 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 922/1860] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:51:22.910159 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6cc6f86f-caab-4e32-9e96-8f524182dd50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c58f05b4afb4181a3ec0abda7418752'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:22.939348 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6cc6f86f-caab-4e32-9e96-8f524182dd50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c58f05b4afb4181a3ec0abda7418752'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:22.942003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6cc6f86f-caab-4e32-9e96-8f524182dd50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c58f05b4afb4181a3ec0abda7418752'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:22.943200 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6cc6f86f-caab-4e32-9e96-8f524182dd50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c58f05b4afb4181a3ec0abda7418752'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:22.967548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6cc6f86f-caab-4e32-9e96-8f524182dd50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c58f05b4afb4181a3ec0abda7418752'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:23.114172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 939/1861] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:22 2022] DELETE /identity/v3/users/2c58f05b4afb4181a3ec0abda7418752 => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:51:23.117818 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd069e-a64d-44c4-bb8b-50dd213bfe70 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:23.118991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd069e-a64d-44c4-bb8b-50dd213bfe70 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:23.119557 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd069e-a64d-44c4-bb8b-50dd213bfe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:23.201336 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0dcd069e-a64d-44c4-bb8b-50dd213bfe70 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:23.207086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0dcd069e-a64d-44c4-bb8b-50dd213bfe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:23.421626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 923/1862] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:23.459410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6f86f-caab-4e32-9e96-8f524182dd50 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:23.466004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6cc6f86f-caab-4e32-9e96-8f524182dd50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:23.689704 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6f86f-caab-4e32-9e96-8f524182dd50 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:23.707747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f0c09-b177-4c9f-b9d4-7707216d4e17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:23.708678 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f0c09-b177-4c9f-b9d4-7707216d4e17 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:23.708963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f0c09-b177-4c9f-b9d4-7707216d4e17 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:23.711461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f0c09-b177-4c9f-b9d4-7707216d4e17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:23.737819 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f0c09-b177-4c9f-b9d4-7707216d4e17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:23.739609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f0c09-b177-4c9f-b9d4-7707216d4e17 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:23.777780 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 940/1863] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:23.885686 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd069e-a64d-44c4-bb8b-50dd213bfe70 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:23.889455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0dcd069e-a64d-44c4-bb8b-50dd213bfe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:23.974825 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd069e-a64d-44c4-bb8b-50dd213bfe70 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:23.984634 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:23.991957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:23.991957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:23.991957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] PATH_INFO: `/v3/projects/ab033e1c6d8146d4a1a873a4d40393a9` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:23.991957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab033e1c6d8146d4a1a873a4d40393a9)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:23.997044 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:24.089869 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:24.204098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab033e1c6d8146d4a1a873a4d40393a9', 'request_id': 'req-1bda7079-bc26-45d8-a313-449df715d6e8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:24.215511 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab033e1c6d8146d4a1a873a4d40393a9', 'request_id': 'req-1bda7079-bc26-45d8-a313-449df715d6e8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:24.237653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab033e1c6d8146d4a1a873a4d40393a9', 'request_id': 'req-1bda7079-bc26-45d8-a313-449df715d6e8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:24.270014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] The token cache is being invalidate because project ab033e1c6d8146d4a1a873a4d40393a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:24.271402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:24.273045 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:24.276624 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 924/1864] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:23 2022] DELETE /identity/v3/projects/ab033e1c6d8146d4a1a873a4d40393a9 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:24.281989 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f0c09-b177-4c9f-b9d4-7707216d4e17 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:24.326661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f0c09-b177-4c9f-b9d4-7707216d4e17 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:24.543736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5009876-0bc1-4a70-8893-a920a1e69feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:24.544888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5009876-0bc1-4a70-8893-a920a1e69feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:24.545424 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5009876-0bc1-4a70-8893-a920a1e69feb admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:24.546009 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5009876-0bc1-4a70-8893-a920a1e69feb admin admin] PATH_INFO: `/v3/users/396a88dc07cc47c9920699794601e5c6` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:24.548156 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5009876-0bc1-4a70-8893-a920a1e69feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=396a88dc07cc47c9920699794601e5c6)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:24.550828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5009876-0bc1-4a70-8893-a920a1e69feb admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:24.709134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:24.740991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c5009876-0bc1-4a70-8893-a920a1e69feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '396a88dc07cc47c9920699794601e5c6'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:24.755509 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c5009876-0bc1-4a70-8893-a920a1e69feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '396a88dc07cc47c9920699794601e5c6'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:24.757060 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c5009876-0bc1-4a70-8893-a920a1e69feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '396a88dc07cc47c9920699794601e5c6'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:24.759562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c5009876-0bc1-4a70-8893-a920a1e69feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '396a88dc07cc47c9920699794601e5c6'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:24.778048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c5009876-0bc1-4a70-8893-a920a1e69feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '396a88dc07cc47c9920699794601e5c6'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:24.803927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 941/1865] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:24 2022] DELETE /identity/v3/users/396a88dc07cc47c9920699794601e5c6 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:25.038476 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:25.041150 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:25.142296 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda7079-bc26-45d8-a313-449df715d6e8 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:25.373816 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:25.377512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:25.378445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:25.378926 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] PATH_INFO: `/v3/projects/eb9120906e6140c7bbc978eb051e887f` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:25.381578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb9120906e6140c7bbc978eb051e887f)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:25.385336 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:25.529634 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb9120906e6140c7bbc978eb051e887f', 'request_id': 'req-3987eeaa-7017-4d00-9d60-8bd145d6a520'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:25.592782 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb9120906e6140c7bbc978eb051e887f', 'request_id': 'req-3987eeaa-7017-4d00-9d60-8bd145d6a520'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:25.640142 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb9120906e6140c7bbc978eb051e887f', 'request_id': 'req-3987eeaa-7017-4d00-9d60-8bd145d6a520'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:25.688493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] The token cache is being invalidate because project eb9120906e6140c7bbc978eb051e887f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:25.689318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:25.691331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:25.694953 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 925/1866] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:25 2022] DELETE /identity/v3/projects/eb9120906e6140c7bbc978eb051e887f => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:51:25.702631 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f657c-69b0-4488-ab6c-1f6311b62cc1 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:25.703743 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f657c-69b0-4488-ab6c-1f6311b62cc1 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:25.703743 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f657c-69b0-4488-ab6c-1f6311b62cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:25.791186 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-723f657c-69b0-4488-ab6c-1f6311b62cc1 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:25.794209 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-723f657c-69b0-4488-ab6c-1f6311b62cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:26.031539 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:26.033908 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:26.079661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 942/1867] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:26.087916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e175dd-2289-4ef7-a73c-b049f9771aee None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:26.091388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e175dd-2289-4ef7-a73c-b049f9771aee None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:26.091527 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e175dd-2289-4ef7-a73c-b049f9771aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:26.163731 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-78e175dd-2289-4ef7-a73c-b049f9771aee None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:26.166427 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-78e175dd-2289-4ef7-a73c-b049f9771aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:26.183697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:26.190288 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 943/1868] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:26.204244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e175dd-2289-4ef7-a73c-b049f9771aee None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:26.217417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e175dd-2289-4ef7-a73c-b049f9771aee None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:26.223630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d6cb6-7e30-4067-a3cb-8567e542ea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:26.224670 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d6cb6-7e30-4067-a3cb-8567e542ea86 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:26.225095 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d6cb6-7e30-4067-a3cb-8567e542ea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:26.225287 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d6cb6-7e30-4067-a3cb-8567e542ea86 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:26.225957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d6cb6-7e30-4067-a3cb-8567e542ea86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:51:26.226359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d6cb6-7e30-4067-a3cb-8567e542ea86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:26.228618 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d6cb6-7e30-4067-a3cb-8567e542ea86 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:26.248707 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 944/1869] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:51:26.254475 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d6cb6-7e30-4067-a3cb-8567e542ea86 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:26.284253 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d6cb6-7e30-4067-a3cb-8567e542ea86 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:26.299358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987eeaa-7017-4d00-9d60-8bd145d6a520 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:26.311655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdbcfb-1b87-43d3-83b3-5dfe1ffb2105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:26.312511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdbcfb-1b87-43d3-83b3-5dfe1ffb2105 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:26.312875 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdbcfb-1b87-43d3-83b3-5dfe1ffb2105 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:26.313384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdbcfb-1b87-43d3-83b3-5dfe1ffb2105 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:26.314136 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdbcfb-1b87-43d3-83b3-5dfe1ffb2105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:26.316715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdbcfb-1b87-43d3-83b3-5dfe1ffb2105 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:26.318805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d73af5-5315-4fb4-8032-fdd41b834b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:26.319447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73af5-5315-4fb4-8032-fdd41b834b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:26.319723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73af5-5315-4fb4-8032-fdd41b834b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:26.320020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73af5-5315-4fb4-8032-fdd41b834b16 admin admin] PATH_INFO: `/v3/projects/5a33641d25bb4c18bec2f60393b571b1` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:26.321768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d73af5-5315-4fb4-8032-fdd41b834b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a33641d25bb4c18bec2f60393b571b1)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:26.323832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d73af5-5315-4fb4-8032-fdd41b834b16 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:26.382595 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 945/1870] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:51:26 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:26.383663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdbcfb-1b87-43d3-83b3-5dfe1ffb2105 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:26.473808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-51d73af5-5315-4fb4-8032-fdd41b834b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a33641d25bb4c18bec2f60393b571b1', 'request_id': 'req-51d73af5-5315-4fb4-8032-fdd41b834b16'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:26.493742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-51d73af5-5315-4fb4-8032-fdd41b834b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a33641d25bb4c18bec2f60393b571b1', 'request_id': 'req-51d73af5-5315-4fb4-8032-fdd41b834b16'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:26.515048 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-51d73af5-5315-4fb4-8032-fdd41b834b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a33641d25bb4c18bec2f60393b571b1', 'request_id': 'req-51d73af5-5315-4fb4-8032-fdd41b834b16'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:26.543517 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-51d73af5-5315-4fb4-8032-fdd41b834b16 admin admin] The token cache is being invalidate because project 5a33641d25bb4c18bec2f60393b571b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:26.543851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-51d73af5-5315-4fb4-8032-fdd41b834b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:26.548591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-51d73af5-5315-4fb4-8032-fdd41b834b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:26.557379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 926/1871] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:26 2022] DELETE /identity/v3/projects/5a33641d25bb4c18bec2f60393b571b1 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:26.559332 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99405240-d1f9-40df-875c-ed6f21d7a70c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:26.559620 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99405240-d1f9-40df-875c-ed6f21d7a70c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:26.559917 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99405240-d1f9-40df-875c-ed6f21d7a70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:26.564579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdbcfb-1b87-43d3-83b3-5dfe1ffb2105 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:26.589253 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:26.634828 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-99405240-d1f9-40df-875c-ed6f21d7a70c None None] MFA Rules not processed for user `ed6e530c3f544748bd3ffc256b1630b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:26.636859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-99405240-d1f9-40df-875c-ed6f21d7a70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:26.684074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a088a3-a5cf-43ec-b8d1-db392d3190f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:26.684769 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a088a3-a5cf-43ec-b8d1-db392d3190f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:26.685045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a088a3-a5cf-43ec-b8d1-db392d3190f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:26.685314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a088a3-a5cf-43ec-b8d1-db392d3190f4 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:26.685961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a088a3-a5cf-43ec-b8d1-db392d3190f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:26.689068 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a088a3-a5cf-43ec-b8d1-db392d3190f4 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:26.734784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 946/1872] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:51:26 2022] POST /identity/v3/users => generated 287 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:26.736331 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8a401-493a-4737-91f8-94f493414920 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:26.736612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8a401-493a-4737-91f8-94f493414920 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:26.736875 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8a401-493a-4737-91f8-94f493414920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:26.796995 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-c1e8a401-493a-4737-91f8-94f493414920 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:26.798141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c1e8a401-493a-4737-91f8-94f493414920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:26.808825 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 947/1873] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:26.810132 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8a401-493a-4737-91f8-94f493414920 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:26.811769 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c1e8a401-493a-4737-91f8-94f493414920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:26.815444 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 927/1874] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:26.821029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99405240-d1f9-40df-875c-ed6f21d7a70c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:26.826553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-99405240-d1f9-40df-875c-ed6f21d7a70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:26.836172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8a401-493a-4737-91f8-94f493414920 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:26.840604 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b735d-7801-4e4f-b298-cb8020e7aaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:26.841074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b735d-7801-4e4f-b298-cb8020e7aaae admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:26.841074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b735d-7801-4e4f-b298-cb8020e7aaae admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:26.841512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b735d-7801-4e4f-b298-cb8020e7aaae admin admin] PATH_INFO: `/v3/users/75940736541747ccaf166649f1e9a3e8` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:26.844498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b735d-7801-4e4f-b298-cb8020e7aaae admin admin] RBAC: Authorizing `identity:delete_user(user_id=75940736541747ccaf166649f1e9a3e8)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:26.848020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b735d-7801-4e4f-b298-cb8020e7aaae admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:26.886832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99405240-d1f9-40df-875c-ed6f21d7a70c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:26.903269 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dbfe6-36e4-45d8-abbc-284e7de842ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:26.904435 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dbfe6-36e4-45d8-abbc-284e7de842ff admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:26.904877 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dbfe6-36e4-45d8-abbc-284e7de842ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:26.905331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dbfe6-36e4-45d8-abbc-284e7de842ff admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:26.906628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8dbfe6-36e4-45d8-abbc-284e7de842ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:26.908510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8dbfe6-36e4-45d8-abbc-284e7de842ff admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:26.985408 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 928/1875] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:26.987233 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dbfe6-36e4-45d8-abbc-284e7de842ff admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:26.994782 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ba8dbfe6-36e4-45d8-abbc-284e7de842ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:27.026495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7d0b735d-7801-4e4f-b298-cb8020e7aaae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75940736541747ccaf166649f1e9a3e8'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:27.034734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dbfe6-36e4-45d8-abbc-284e7de842ff admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:27.054727 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5cd20-3d4e-4af5-b08c-657d2d7781e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:27.055163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5cd20-3d4e-4af5-b08c-657d2d7781e7 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:27.055437 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5cd20-3d4e-4af5-b08c-657d2d7781e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:27.055707 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5cd20-3d4e-4af5-b08c-657d2d7781e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:27.056510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5cd20-3d4e-4af5-b08c-657d2d7781e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:27.058928 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5cd20-3d4e-4af5-b08c-657d2d7781e7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:27.060793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7d0b735d-7801-4e4f-b298-cb8020e7aaae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75940736541747ccaf166649f1e9a3e8'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:27.064343 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7d0b735d-7801-4e4f-b298-cb8020e7aaae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75940736541747ccaf166649f1e9a3e8'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:27.066947 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7d0b735d-7801-4e4f-b298-cb8020e7aaae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75940736541747ccaf166649f1e9a3e8'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:27.091363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-7d0b735d-7801-4e4f-b298-cb8020e7aaae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75940736541747ccaf166649f1e9a3e8'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:27.159206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 929/1876] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:51:26 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:27.161831 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d077e3-127a-41fc-8bac-dfbb75261893 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:27.162307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d077e3-127a-41fc-8bac-dfbb75261893 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:27.162787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d077e3-127a-41fc-8bac-dfbb75261893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:27.221977 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-94d077e3-127a-41fc-8bac-dfbb75261893 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:27.225610 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-94d077e3-127a-41fc-8bac-dfbb75261893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:27.236425 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 948/1877] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:26 2022] DELETE /identity/v3/users/75940736541747ccaf166649f1e9a3e8 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:27.248315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b735d-7801-4e4f-b298-cb8020e7aaae admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:27.251543 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7d0b735d-7801-4e4f-b298-cb8020e7aaae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:27.260104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:27.365667 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b735d-7801-4e4f-b298-cb8020e7aaae admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:27.372289 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a389c-d621-4811-a82b-f111a6031c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:27.373037 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a389c-d621-4811-a82b-f111a6031c38 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:27.373381 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a389c-d621-4811-a82b-f111a6031c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:27.373718 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a389c-d621-4811-a82b-f111a6031c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:27.375134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a389c-d621-4811-a82b-f111a6031c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:27.377146 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a389c-d621-4811-a82b-f111a6031c38 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:27.390914 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 949/1878] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:27.391627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a389c-d621-4811-a82b-f111a6031c38 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:27.404547 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1a4a389c-d621-4811-a82b-f111a6031c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:27.404826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 930/1879] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:27.405082 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e44a4-19e0-4890-9b30-1a76145fd04a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:27.406188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e44a4-19e0-4890-9b30-1a76145fd04a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:27.406327 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e44a4-19e0-4890-9b30-1a76145fd04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:27.437847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a389c-d621-4811-a82b-f111a6031c38 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:27.456249 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b28e44a4-19e0-4890-9b30-1a76145fd04a None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:27.456249 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b28e44a4-19e0-4890-9b30-1a76145fd04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:27.470334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd04ea0-1a08-40e5-b5f5-406b53468a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:27.472381 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd04ea0-1a08-40e5-b5f5-406b53468a29 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:27.473477 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd04ea0-1a08-40e5-b5f5-406b53468a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:27.474224 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd04ea0-1a08-40e5-b5f5-406b53468a29 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:27.475752 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd04ea0-1a08-40e5-b5f5-406b53468a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:27.483619 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd04ea0-1a08-40e5-b5f5-406b53468a29 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:27.498640 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 931/1880] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:27.502084 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:27.503439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e44a4-19e0-4890-9b30-1a76145fd04a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:27.536558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e44a4-19e0-4890-9b30-1a76145fd04a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:27.540291 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 950/1881] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:51:27 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:27.541951 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd04ea0-1a08-40e5-b5f5-406b53468a29 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:27.548400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55093a1-af03-45d5-aa11-81b2f53547ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:27.549290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55093a1-af03-45d5-aa11-81b2f53547ce admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:27.549631 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55093a1-af03-45d5-aa11-81b2f53547ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:27.550013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55093a1-af03-45d5-aa11-81b2f53547ce admin admin] PATH_INFO: `/v3/projects/9da8e5e0fae247f48af6d303ae2fcdff/users/a02652d1b503426cac6e77aa68f2e98c/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:27.558663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd04ea0-1a08-40e5-b5f5-406b53468a29 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:27.565805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9535c5-819a-4a22-88ac-ef6cccf554e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:27.566570 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9535c5-819a-4a22-88ac-ef6cccf554e9 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:27.567062 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9535c5-819a-4a22-88ac-ef6cccf554e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:27.567595 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9535c5-819a-4a22-88ac-ef6cccf554e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:27.568276 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9535c5-819a-4a22-88ac-ef6cccf554e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:51:27.568616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9535c5-819a-4a22-88ac-ef6cccf554e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:27.571007 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9535c5-819a-4a22-88ac-ef6cccf554e9 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:27.581447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55093a1-af03-45d5-aa11-81b2f53547ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=9da8e5e0fae247f48af6d303ae2fcdff, user_id=a02652d1b503426cac6e77aa68f2e98c, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:27.584770 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55093a1-af03-45d5-aa11-81b2f53547ce admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:27.591494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 951/1882] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:51:27.592516 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9535c5-819a-4a22-88ac-ef6cccf554e9 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:27.595152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4b9535c5-819a-4a22-88ac-ef6cccf554e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:27.610430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 932/1883] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:27 2022] PUT /identity/v3/projects/9da8e5e0fae247f48af6d303ae2fcdff/users/a02652d1b503426cac6e77aa68f2e98c/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:27.611794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55093a1-af03-45d5-aa11-81b2f53547ce admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:27.801648 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55093a1-af03-45d5-aa11-81b2f53547ce admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:27.803699 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9535c5-819a-4a22-88ac-ef6cccf554e9 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:27.823836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860001f6-0090-447e-9022-b41a62766a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:27.824879 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860001f6-0090-447e-9022-b41a62766a93 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:27.825333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860001f6-0090-447e-9022-b41a62766a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:27.825780 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860001f6-0090-447e-9022-b41a62766a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:27.826774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860001f6-0090-447e-9022-b41a62766a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:27.838171 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860001f6-0090-447e-9022-b41a62766a93 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:27.879997 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 933/1884] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:27.881017 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860001f6-0090-447e-9022-b41a62766a93 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:27.913708 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860001f6-0090-447e-9022-b41a62766a93 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:27.923404 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d88a37-8a3e-4978-9463-0bf8b42b42ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:27.924447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d88a37-8a3e-4978-9463-0bf8b42b42ef admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:27.925201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d88a37-8a3e-4978-9463-0bf8b42b42ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:27.925693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d88a37-8a3e-4978-9463-0bf8b42b42ef admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:27.926771 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d88a37-8a3e-4978-9463-0bf8b42b42ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:27.933286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d88a37-8a3e-4978-9463-0bf8b42b42ef admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:27.991029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde51dea-6ba2-4299-9bb7-21397da4b33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:27.992286 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde51dea-6ba2-4299-9bb7-21397da4b33b service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:27.992773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde51dea-6ba2-4299-9bb7-21397da4b33b service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:27.993254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde51dea-6ba2-4299-9bb7-21397da4b33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:28.016626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 934/1885] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:51:27 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:28.018627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde51dea-6ba2-4299-9bb7-21397da4b33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:28.019156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139b67b-09ab-495b-a839-cb2122a5b5a0 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:28.019565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139b67b-09ab-495b-a839-cb2122a5b5a0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:28.020081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139b67b-09ab-495b-a839-cb2122a5b5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:28.022819 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde51dea-6ba2-4299-9bb7-21397da4b33b service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:28.044450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 952/1886] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:28.046585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde51dea-6ba2-4299-9bb7-21397da4b33b service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:28.076674 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde51dea-6ba2-4299-9bb7-21397da4b33b service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:28.156055 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f139b67b-09ab-495b-a839-cb2122a5b5a0 None None] MFA Rules not processed for user `a02652d1b503426cac6e77aa68f2e98c`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:28.158111 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f139b67b-09ab-495b-a839-cb2122a5b5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:28.289384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0b6ec-1c77-49d5-843c-6ac9f4e33b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:28.295603 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0b6ec-1c77-49d5-843c-6ac9f4e33b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:28.298007 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0b6ec-1c77-49d5-843c-6ac9f4e33b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:28.298919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0b6ec-1c77-49d5-843c-6ac9f4e33b35 admin admin] PATH_INFO: `/v3/projects/d5b8e9a790184f12b5188156c43053a1/users/4e1de7598b0f40a7b844879adac1a39b/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:28.348875 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:28.350411 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0b6ec-1c77-49d5-843c-6ac9f4e33b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b8e9a790184f12b5188156c43053a1, user_id=4e1de7598b0f40a7b844879adac1a39b, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:28.354397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0b6ec-1c77-49d5-843c-6ac9f4e33b35 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:28.379844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 935/1887] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:28.381114 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139b67b-09ab-495b-a839-cb2122a5b5a0 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:28.400335 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 953/1888] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:28 2022] PUT /identity/v3/projects/d5b8e9a790184f12b5188156c43053a1/users/4e1de7598b0f40a7b844879adac1a39b/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:28.401134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0b6ec-1c77-49d5-843c-6ac9f4e33b35 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:28.405111 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139b67b-09ab-495b-a839-cb2122a5b5a0 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:28.405388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9cd0b6ec-1c77-49d5-843c-6ac9f4e33b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:28.463776 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:28.548033 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0b6ec-1c77-49d5-843c-6ac9f4e33b35 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:28.552469 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ba462-5dc5-447e-8b3e-dd9e72b987f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:28.553381 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ba462-5dc5-447e-8b3e-dd9e72b987f3 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:28.553880 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ba462-5dc5-447e-8b3e-dd9e72b987f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:28.554227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ba462-5dc5-447e-8b3e-dd9e72b987f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:28.562546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49707cda-45f7-4208-bceb-193a40e38b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:28.563974 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49707cda-45f7-4208-bceb-193a40e38b51 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:28.564455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49707cda-45f7-4208-bceb-193a40e38b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:28.564995 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49707cda-45f7-4208-bceb-193a40e38b51 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:28.566180 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49707cda-45f7-4208-bceb-193a40e38b51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:28.570281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49707cda-45f7-4208-bceb-193a40e38b51 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:28.621666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 936/1889] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:51:28 2022] POST /identity/v3/users => generated 285 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:28.621666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49707cda-45f7-4208-bceb-193a40e38b51 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:28.641582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ba462-5dc5-447e-8b3e-dd9e72b987f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:28.644136 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ba462-5dc5-447e-8b3e-dd9e72b987f3 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:28.650502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49707cda-45f7-4208-bceb-193a40e38b51 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:28.660527 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 954/1890] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:28.661494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ba462-5dc5-447e-8b3e-dd9e72b987f3 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:28.666502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b691e-604e-496c-ab9f-f5b9d64c1206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:28.666932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b691e-604e-496c-ab9f-f5b9d64c1206 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:28.667217 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b691e-604e-496c-ab9f-f5b9d64c1206 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:28.667528 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b691e-604e-496c-ab9f-f5b9d64c1206 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:28.668255 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b691e-604e-496c-ab9f-f5b9d64c1206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:28.670021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b691e-604e-496c-ab9f-f5b9d64c1206 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:28.684146 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ba462-5dc5-447e-8b3e-dd9e72b987f3 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:28.694870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dad15e-b6be-4d54-966e-dc3192eb1b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:28.696575 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dad15e-b6be-4d54-966e-dc3192eb1b77 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:28.697269 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dad15e-b6be-4d54-966e-dc3192eb1b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:28.697849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dad15e-b6be-4d54-966e-dc3192eb1b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:28.699426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dad15e-b6be-4d54-966e-dc3192eb1b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:28.703507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dad15e-b6be-4d54-966e-dc3192eb1b77 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:28.708757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 937/1891] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:28.710301 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b691e-604e-496c-ab9f-f5b9d64c1206 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:28.715674 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-518b691e-604e-496c-ab9f-f5b9d64c1206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:28.721020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 955/1892] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:28.724731 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dad15e-b6be-4d54-966e-dc3192eb1b77 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:28.738960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dad15e-b6be-4d54-966e-dc3192eb1b77 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:28.746181 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d67b46-0cea-4a3c-abc7-3aca9455dfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:28.746879 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d67b46-0cea-4a3c-abc7-3aca9455dfcc admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:28.747152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d67b46-0cea-4a3c-abc7-3aca9455dfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:28.747419 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d67b46-0cea-4a3c-abc7-3aca9455dfcc admin admin] PATH_INFO: `/v3/projects/d5b8e9a790184f12b5188156c43053a1/users/4e1de7598b0f40a7b844879adac1a39b/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:28.750007 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d67b46-0cea-4a3c-abc7-3aca9455dfcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b8e9a790184f12b5188156c43053a1, user_id=4e1de7598b0f40a7b844879adac1a39b, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:28.752886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d67b46-0cea-4a3c-abc7-3aca9455dfcc admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:28.753201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b691e-604e-496c-ab9f-f5b9d64c1206 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:28.764118 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad93a2c-f472-4395-b921-92d3b78620c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:28.765915 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93a2c-f472-4395-b921-92d3b78620c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:28.766401 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93a2c-f472-4395-b921-92d3b78620c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:28.766882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93a2c-f472-4395-b921-92d3b78620c1 admin admin] PATH_INFO: `/v3/projects/8839649043e14e588ab184df6e2252c8` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:28.768249 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 956/1893] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:28 2022] PUT /identity/v3/projects/d5b8e9a790184f12b5188156c43053a1/users/4e1de7598b0f40a7b844879adac1a39b/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:28.769276 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d67b46-0cea-4a3c-abc7-3aca9455dfcc admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:28.769796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad93a2c-f472-4395-b921-92d3b78620c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8839649043e14e588ab184df6e2252c8)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:28.773377 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad93a2c-f472-4395-b921-92d3b78620c1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:28.891913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d67b46-0cea-4a3c-abc7-3aca9455dfcc admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:28.907484 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1552f-ea00-4254-8809-2d6751c58ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:28.908928 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1552f-ea00-4254-8809-2d6751c58ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:28.909208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1552f-ea00-4254-8809-2d6751c58ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:28.914184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1552f-ea00-4254-8809-2d6751c58ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:28.917488 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1552f-ea00-4254-8809-2d6751c58ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:28.929376 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1552f-ea00-4254-8809-2d6751c58ae3 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:28.967276 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 957/1894] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:28.969796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1552f-ea00-4254-8809-2d6751c58ae3 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:28.981917 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0ad93a2c-f472-4395-b921-92d3b78620c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8839649043e14e588ab184df6e2252c8', 'request_id': 'req-0ad93a2c-f472-4395-b921-92d3b78620c1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:29.001528 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1552f-ea00-4254-8809-2d6751c58ae3 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:29.010118 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0ad93a2c-f472-4395-b921-92d3b78620c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8839649043e14e588ab184df6e2252c8', 'request_id': 'req-0ad93a2c-f472-4395-b921-92d3b78620c1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:29.027311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0ad93a2c-f472-4395-b921-92d3b78620c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8839649043e14e588ab184df6e2252c8', 'request_id': 'req-0ad93a2c-f472-4395-b921-92d3b78620c1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:29.053547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0ad93a2c-f472-4395-b921-92d3b78620c1 admin admin] The token cache is being invalidate because project 8839649043e14e588ab184df6e2252c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:29.053963 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0ad93a2c-f472-4395-b921-92d3b78620c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:29.056033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0ad93a2c-f472-4395-b921-92d3b78620c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:29.057665 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 938/1895] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:28 2022] DELETE /identity/v3/projects/8839649043e14e588ab184df6e2252c8 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:29.065743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718a170-db06-4956-ae6c-71b95bc490a1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:29.066081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718a170-db06-4956-ae6c-71b95bc490a1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:29.066357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718a170-db06-4956-ae6c-71b95bc490a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:29.104205 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b718a170-db06-4956-ae6c-71b95bc490a1 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:29.105973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b718a170-db06-4956-ae6c-71b95bc490a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:29.193290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:29.238900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:29.239189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33216077-e126-4347-8092-a8eb4023004c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:29.240079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33216077-e126-4347-8092-a8eb4023004c admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:29.240222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33216077-e126-4347-8092-a8eb4023004c admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:29.240417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33216077-e126-4347-8092-a8eb4023004c admin admin] PATH_INFO: `/v3/projects/0c7a2638819b4f638aa9c4113b3838b5/users/8459b76bcc23475f821cd78548876bc2/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:29.269459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 939/1896] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:29.285353 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e8b6b-eddc-4f81-bc63-fcc7ef8a1937 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:29.285793 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e8b6b-eddc-4f81-bc63-fcc7ef8a1937 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:29.286352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e8b6b-eddc-4f81-bc63-fcc7ef8a1937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:29.323906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33216077-e126-4347-8092-a8eb4023004c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c7a2638819b4f638aa9c4113b3838b5, user_id=8459b76bcc23475f821cd78548876bc2, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:29.323906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33216077-e126-4347-8092-a8eb4023004c admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:29.380278 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 958/1897] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:28 2022] PUT /identity/v3/projects/0c7a2638819b4f638aa9c4113b3838b5/users/8459b76bcc23475f821cd78548876bc2/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:51:29.383097 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-fe2e8b6b-eddc-4f81-bc63-fcc7ef8a1937 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:29.384402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fe2e8b6b-eddc-4f81-bc63-fcc7ef8a1937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:29.386491 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4037d55-cf48-40e4-a28e-2519ccae76ea None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:29.386829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4037d55-cf48-40e4-a28e-2519ccae76ea None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:29.387155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4037d55-cf48-40e4-a28e-2519ccae76ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:29.471378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-c4037d55-cf48-40e4-a28e-2519ccae76ea None None] MFA Rules not processed for user `8459b76bcc23475f821cd78548876bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:29.471378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c4037d55-cf48-40e4-a28e-2519ccae76ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:29.613601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 940/1898] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:29.617982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e8b6b-eddc-4f81-bc63-fcc7ef8a1937 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:29.632642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e8b6b-eddc-4f81-bc63-fcc7ef8a1937 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:29.638901 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 959/1899] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:29.644669 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4037d55-cf48-40e4-a28e-2519ccae76ea None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:29.647358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c4037d55-cf48-40e4-a28e-2519ccae76ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:29.652301 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5ac11-77f8-45c1-b434-259bd23d26d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:29.652960 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5ac11-77f8-45c1-b434-259bd23d26d0 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:29.653243 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5ac11-77f8-45c1-b434-259bd23d26d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:29.653516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5ac11-77f8-45c1-b434-259bd23d26d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:29.654164 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5ac11-77f8-45c1-b434-259bd23d26d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:51:29.654466 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5ac11-77f8-45c1-b434-259bd23d26d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:29.656361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5ac11-77f8-45c1-b434-259bd23d26d0 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:29.668208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4037d55-cf48-40e4-a28e-2519ccae76ea None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:29.676400 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17591c6-9ca3-40f5-aa34-ea4ee465a78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:29.677389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17591c6-9ca3-40f5-aa34-ea4ee465a78e admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:29.677632 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17591c6-9ca3-40f5-aa34-ea4ee465a78e admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:29.678040 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17591c6-9ca3-40f5-aa34-ea4ee465a78e admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:29.678931 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17591c6-9ca3-40f5-aa34-ea4ee465a78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:29.682032 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17591c6-9ca3-40f5-aa34-ea4ee465a78e admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:29.712696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 941/1900] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:51:29.713769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5ac11-77f8-45c1-b434-259bd23d26d0 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:29.738708 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5ac11-77f8-45c1-b434-259bd23d26d0 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:29.745468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 960/1901] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:51:29 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:51:29.750067 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17591c6-9ca3-40f5-aa34-ea4ee465a78e admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:29.765753 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86684dc1-68c6-4923-89c6-f6083fe1fdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:29.766416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86684dc1-68c6-4923-89c6-f6083fe1fdfe admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:29.766687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86684dc1-68c6-4923-89c6-f6083fe1fdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:29.766972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86684dc1-68c6-4923-89c6-f6083fe1fdfe admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:29.767622 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86684dc1-68c6-4923-89c6-f6083fe1fdfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:29.769690 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86684dc1-68c6-4923-89c6-f6083fe1fdfe admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:29.846182 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 942/1902] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:51:29 2022] POST /identity/v3/projects => generated 369 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:29.850503 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86684dc1-68c6-4923-89c6-f6083fe1fdfe admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:29.903913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17591c6-9ca3-40f5-aa34-ea4ee465a78e admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:29.993699 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:29.997102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86684dc1-68c6-4923-89c6-f6083fe1fdfe admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:30.008445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182efa7-c28a-4c60-81ba-510855155257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:30.009971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182efa7-c28a-4c60-81ba-510855155257 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:30.010446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182efa7-c28a-4c60-81ba-510855155257 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:30.011508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182efa7-c28a-4c60-81ba-510855155257 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:30.012797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182efa7-c28a-4c60-81ba-510855155257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:30.018076 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182efa7-c28a-4c60-81ba-510855155257 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:30.037396 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8797e0-5145-4562-8df0-9b3a718decad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:30.038111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8797e0-5145-4562-8df0-9b3a718decad admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:30.039097 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8797e0-5145-4562-8df0-9b3a718decad admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:30.039329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8797e0-5145-4562-8df0-9b3a718decad admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:30.040401 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8797e0-5145-4562-8df0-9b3a718decad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:30.063355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8797e0-5145-4562-8df0-9b3a718decad admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:30.103377 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 943/1903] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:51:29 2022] POST /identity/v3/users => generated 285 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:30.108905 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182efa7-c28a-4c60-81ba-510855155257 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:30.134739 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182efa7-c28a-4c60-81ba-510855155257 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:30.137533 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 961/1904] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:51:29 2022] POST /identity/v3/users => generated 290 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:30.143391 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8797e0-5145-4562-8df0-9b3a718decad admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:30.152658 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42988858-1ecf-4fdd-ab64-52992e3fea80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:30.153337 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42988858-1ecf-4fdd-ab64-52992e3fea80 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:30.153687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42988858-1ecf-4fdd-ab64-52992e3fea80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:30.153884 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42988858-1ecf-4fdd-ab64-52992e3fea80 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:30.154505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42988858-1ecf-4fdd-ab64-52992e3fea80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:30.158902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42988858-1ecf-4fdd-ab64-52992e3fea80 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:30.176985 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 944/1905] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:30.181050 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42988858-1ecf-4fdd-ab64-52992e3fea80 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:30.185485 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8797e0-5145-4562-8df0-9b3a718decad admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:30.196463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6f708-7328-4544-bae4-965dc77c1d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:30.196767 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42988858-1ecf-4fdd-ab64-52992e3fea80 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:30.197542 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6f708-7328-4544-bae4-965dc77c1d08 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:30.198010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6f708-7328-4544-bae4-965dc77c1d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:30.198673 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6f708-7328-4544-bae4-965dc77c1d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:30.199701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b6f708-7328-4544-bae4-965dc77c1d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:30.204755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b6f708-7328-4544-bae4-965dc77c1d08 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:30.205353 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcabf6e-8dcb-4df0-886c-c979efb1ac66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:30.206290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcabf6e-8dcb-4df0-886c-c979efb1ac66 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:30.207090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcabf6e-8dcb-4df0-886c-c979efb1ac66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:30.207529 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcabf6e-8dcb-4df0-886c-c979efb1ac66 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:30.208616 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcabf6e-8dcb-4df0-886c-c979efb1ac66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:30.211232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcabf6e-8dcb-4df0-886c-c979efb1ac66 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:30.234223 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 945/1906] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:30.234503 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 962/1907] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:51:30.237503 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcabf6e-8dcb-4df0-886c-c979efb1ac66 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:30.239343 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6f708-7328-4544-bae4-965dc77c1d08 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:30.251132 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcabf6e-8dcb-4df0-886c-c979efb1ac66 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:30.258628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f57bf-6d14-4e71-b086-2b22e9b99c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:30.259470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f57bf-6d14-4e71-b086-2b22e9b99c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:30.259931 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f57bf-6d14-4e71-b086-2b22e9b99c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:30.260209 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f57bf-6d14-4e71-b086-2b22e9b99c30 admin admin] PATH_INFO: `/v3/projects/60710b498def444ab00751cf12ef8940/users/0abbd790650f40a6b89b3179c2078f88/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:30.275019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6f708-7328-4544-bae4-965dc77c1d08 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:30.284815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9301ac0-58fb-4890-9ff8-c6ba711ba9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:30.286090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9301ac0-58fb-4890-9ff8-c6ba711ba9f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:30.286640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9301ac0-58fb-4890-9ff8-c6ba711ba9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:30.288149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9301ac0-58fb-4890-9ff8-c6ba711ba9f7 admin admin] PATH_INFO: `/v3/projects/112fb74ca1ec45e6bff1bf23351d4779/users/87941fe58ba94c0a8715c4f6eb7f35c5/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:30.296490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f57bf-6d14-4e71-b086-2b22e9b99c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60710b498def444ab00751cf12ef8940, user_id=0abbd790650f40a6b89b3179c2078f88, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:30.303656 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f57bf-6d14-4e71-b086-2b22e9b99c30 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:30.325627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9301ac0-58fb-4890-9ff8-c6ba711ba9f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=112fb74ca1ec45e6bff1bf23351d4779, user_id=87941fe58ba94c0a8715c4f6eb7f35c5, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:30.330191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 946/1908] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:30 2022] PUT /identity/v3/projects/60710b498def444ab00751cf12ef8940/users/0abbd790650f40a6b89b3179c2078f88/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:30.334352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9301ac0-58fb-4890-9ff8-c6ba711ba9f7 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:30.343267 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e4e06-4e67-4b66-9c83-8ad6159a3e8d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:30.343979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e4e06-4e67-4b66-9c83-8ad6159a3e8d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:30.344611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e4e06-4e67-4b66-9c83-8ad6159a3e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:30.380573 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 963/1909] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:30 2022] PUT /identity/v3/projects/112fb74ca1ec45e6bff1bf23351d4779/users/87941fe58ba94c0a8715c4f6eb7f35c5/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:30.385246 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9301ac0-58fb-4890-9ff8-c6ba711ba9f7 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:30.389333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:30.427927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-134e4e06-4e67-4b66-9c83-8ad6159a3e8d None None] MFA Rules not processed for user `0abbd790650f40a6b89b3179c2078f88`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:30.432278 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-134e4e06-4e67-4b66-9c83-8ad6159a3e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:30.527261 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9301ac0-58fb-4890-9ff8-c6ba711ba9f7 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:30.535349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4eb8d-239b-4103-b16c-4c102526e22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:30.536498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4eb8d-239b-4103-b16c-4c102526e22b admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:30.536977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4eb8d-239b-4103-b16c-4c102526e22b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:30.537632 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4eb8d-239b-4103-b16c-4c102526e22b admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:30.538935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4eb8d-239b-4103-b16c-4c102526e22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:30.542565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4eb8d-239b-4103-b16c-4c102526e22b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:30.598807 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 964/1910] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:51:30.600359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4eb8d-239b-4103-b16c-4c102526e22b admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:30.612628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 947/1911] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:30.621616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4eb8d-239b-4103-b16c-4c102526e22b admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:30.626193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e4e06-4e67-4b66-9c83-8ad6159a3e8d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:30.629387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-134e4e06-4e67-4b66-9c83-8ad6159a3e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:30.637136 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233e446-da1c-4cc4-b4ba-7b8d061e4082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:30.638554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233e446-da1c-4cc4-b4ba-7b8d061e4082 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:30.639045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233e446-da1c-4cc4-b4ba-7b8d061e4082 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:30.639511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233e446-da1c-4cc4-b4ba-7b8d061e4082 admin admin] PATH_INFO: `/v3/projects/112fb74ca1ec45e6bff1bf23351d4779/users/87941fe58ba94c0a8715c4f6eb7f35c5/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:30.642546 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233e446-da1c-4cc4-b4ba-7b8d061e4082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=112fb74ca1ec45e6bff1bf23351d4779, user_id=87941fe58ba94c0a8715c4f6eb7f35c5, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:30.646667 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233e446-da1c-4cc4-b4ba-7b8d061e4082 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:30.662857 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 965/1912] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:30 2022] PUT /identity/v3/projects/112fb74ca1ec45e6bff1bf23351d4779/users/87941fe58ba94c0a8715c4f6eb7f35c5/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:30.688245 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4f567-b5ca-4d6b-9b86-e5ccfaff96af None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:30.688867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4f567-b5ca-4d6b-9b86-e5ccfaff96af None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:30.694295 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4f567-b5ca-4d6b-9b86-e5ccfaff96af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:30.754362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-fda4f567-b5ca-4d6b-9b86-e5ccfaff96af None None] MFA Rules not processed for user `87941fe58ba94c0a8715c4f6eb7f35c5`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:30.758938 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-fda4f567-b5ca-4d6b-9b86-e5ccfaff96af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:30.764067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e4e06-4e67-4b66-9c83-8ad6159a3e8d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:30.900983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59d52c-b3e6-4806-b888-33dde50352b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f5aabcb1414413a8dce7b76ce701582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:30.902052 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59d52c-b3e6-4806-b888-33dde50352b1 service glance] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:30.904374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59d52c-b3e6-4806-b888-33dde50352b1 service glance] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:30.905193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59d52c-b3e6-4806-b888-33dde50352b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:30.920787 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 966/1913] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:30.935059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4f567-b5ca-4d6b-9b86-e5ccfaff96af None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:30.938753 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-fda4f567-b5ca-4d6b-9b86-e5ccfaff96af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:30.955084 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:31.096183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4f567-b5ca-4d6b-9b86-e5ccfaff96af None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:31.108083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4aac01-bdce-426d-98a6-e75cb83971ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:31.109926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aac01-bdce-426d-98a6-e75cb83971ef service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:31.109926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aac01-bdce-426d-98a6-e75cb83971ef service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:31.110255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aac01-bdce-426d-98a6-e75cb83971ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:31.112226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59d52c-b3e6-4806-b888-33dde50352b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:31.129439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59d52c-b3e6-4806-b888-33dde50352b1 service glance] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:31.157734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4aac01-bdce-426d-98a6-e75cb83971ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:31.162161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4aac01-bdce-426d-98a6-e75cb83971ef service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:31.173533 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 948/1914] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:31.182491 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 967/1915] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:51:31.286232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59d52c-b3e6-4806-b888-33dde50352b1 service glance] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:31.315201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59d52c-b3e6-4806-b888-33dde50352b1 service glance] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:31.323827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c86eb-a110-48e9-9c99-fe4c092634d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:31.324876 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c86eb-a110-48e9-9c99-fe4c092634d1 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:31.325337 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c86eb-a110-48e9-9c99-fe4c092634d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:31.325808 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c86eb-a110-48e9-9c99-fe4c092634d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:31.338245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:31.486754 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c86eb-a110-48e9-9c99-fe4c092634d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:31.488307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c86eb-a110-48e9-9c99-fe4c092634d1 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:31.510125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 949/1916] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:33.600072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83351caa-99fe-4359-b93d-4dd9d5bd7964 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:33.601524 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83351caa-99fe-4359-b93d-4dd9d5bd7964 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:33.601956 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83351caa-99fe-4359-b93d-4dd9d5bd7964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:33.628123 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-83351caa-99fe-4359-b93d-4dd9d5bd7964 None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:33.628531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-83351caa-99fe-4359-b93d-4dd9d5bd7964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:33.796029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 968/1917] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:51:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:33.943970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c86eb-a110-48e9-9c99-fe4c092634d1 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:33.952587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-147c86eb-a110-48e9-9c99-fe4c092634d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:34.240573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c86eb-a110-48e9-9c99-fe4c092634d1 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:34.278722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9164f-d209-416d-ac11-8cf590b16221 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:34.279590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9164f-d209-416d-ac11-8cf590b16221 service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:34.280272 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9164f-d209-416d-ac11-8cf590b16221 service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:34.281048 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9164f-d209-416d-ac11-8cf590b16221 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:34.361961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9164f-d209-416d-ac11-8cf590b16221 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:34.378286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9164f-d209-416d-ac11-8cf590b16221 service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:34.394531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8d0a4-52f6-47d1-831b-16adbfe61f76 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:34.394987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8d0a4-52f6-47d1-831b-16adbfe61f76 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:34.395501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8d0a4-52f6-47d1-831b-16adbfe61f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:34.421538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 950/1918] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:51:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:51:34.482200 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-bdf8d0a4-52f6-47d1-831b-16adbfe61f76 None None] MFA Rules not processed for user `a02652d1b503426cac6e77aa68f2e98c`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:34.483326 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-bdf8d0a4-52f6-47d1-831b-16adbfe61f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:34.680458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 969/1919] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:51:37.025984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9164f-d209-416d-ac11-8cf590b16221 service swift] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:37.140357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9164f-d209-416d-ac11-8cf590b16221 service swift] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:37.147264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52400e-66ce-4976-97f5-7b4a4a4ce793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:37.148529 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52400e-66ce-4976-97f5-7b4a4a4ce793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:37.149095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52400e-66ce-4976-97f5-7b4a4a4ce793 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:37.149607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52400e-66ce-4976-97f5-7b4a4a4ce793 admin admin] PATH_INFO: `/v3/users/8459b76bcc23475f821cd78548876bc2` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:37.152222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52400e-66ce-4976-97f5-7b4a4a4ce793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8459b76bcc23475f821cd78548876bc2)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:37.155939 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52400e-66ce-4976-97f5-7b4a4a4ce793 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:37.292940 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:37.322865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-da52400e-66ce-4976-97f5-7b4a4a4ce793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8459b76bcc23475f821cd78548876bc2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:37.324967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-da52400e-66ce-4976-97f5-7b4a4a4ce793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8459b76bcc23475f821cd78548876bc2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:37.329362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-da52400e-66ce-4976-97f5-7b4a4a4ce793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8459b76bcc23475f821cd78548876bc2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:37.361621 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-da52400e-66ce-4976-97f5-7b4a4a4ce793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8459b76bcc23475f821cd78548876bc2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:37.400097 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-da52400e-66ce-4976-97f5-7b4a4a4ce793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8459b76bcc23475f821cd78548876bc2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:37.512095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 951/1920] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:37 2022] DELETE /identity/v3/users/8459b76bcc23475f821cd78548876bc2 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:51:37.519899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b4674-6946-4d3d-a5c9-e8afe3453b08 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:37.520793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b4674-6946-4d3d-a5c9-e8afe3453b08 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:37.520793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b4674-6946-4d3d-a5c9-e8afe3453b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:37.560422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-a32b4674-6946-4d3d-a5c9-e8afe3453b08 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:37.562905 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-a32b4674-6946-4d3d-a5c9-e8afe3453b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:37.951235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:37.978964 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 970/1921] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:37.987686 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52400e-66ce-4976-97f5-7b4a4a4ce793 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:37.993183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-da52400e-66ce-4976-97f5-7b4a4a4ce793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:38.183856 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52400e-66ce-4976-97f5-7b4a4a4ce793 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:38.207074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641395ac-4444-4123-b145-0c78f9cdda42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:38.207896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641395ac-4444-4123-b145-0c78f9cdda42 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:38.208212 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641395ac-4444-4123-b145-0c78f9cdda42 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:38.208524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641395ac-4444-4123-b145-0c78f9cdda42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:38.225697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641395ac-4444-4123-b145-0c78f9cdda42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:38.230951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641395ac-4444-4123-b145-0c78f9cdda42 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:38.247846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 952/1922] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:38.536300 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f97ea-41d7-47c2-8385-77ddc85ef31d None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:38.537339 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f97ea-41d7-47c2-8385-77ddc85ef31d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:38.537834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f97ea-41d7-47c2-8385-77ddc85ef31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:38.628083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-360f97ea-41d7-47c2-8385-77ddc85ef31d None None] MFA Rules not processed for user `4e1de7598b0f40a7b844879adac1a39b`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:38.630658 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-360f97ea-41d7-47c2-8385-77ddc85ef31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:38.809550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641395ac-4444-4123-b145-0c78f9cdda42 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:38.813214 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-641395ac-4444-4123-b145-0c78f9cdda42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:38.844288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641395ac-4444-4123-b145-0c78f9cdda42 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:38.851067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:38.851777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:38.852160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:38.852319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] PATH_INFO: `/v3/projects/0c7a2638819b4f638aa9c4113b3838b5` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:38.854327 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c7a2638819b4f638aa9c4113b3838b5)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:38.857825 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:38.883685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 971/1923] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:38.927651 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f97ea-41d7-47c2-8385-77ddc85ef31d None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:38.929826 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-360f97ea-41d7-47c2-8385-77ddc85ef31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:38.969340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c7a2638819b4f638aa9c4113b3838b5', 'request_id': 'req-21415f85-1076-471d-a705-a6b33f4f4662'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:38.982447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c7a2638819b4f638aa9c4113b3838b5', 'request_id': 'req-21415f85-1076-471d-a705-a6b33f4f4662'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:39.002064 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c7a2638819b4f638aa9c4113b3838b5', 'request_id': 'req-21415f85-1076-471d-a705-a6b33f4f4662'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:39.030018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] The token cache is being invalidate because project 0c7a2638819b4f638aa9c4113b3838b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:39.032715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:39.054433 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:39.066297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 953/1924] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:38 2022] DELETE /identity/v3/projects/0c7a2638819b4f638aa9c4113b3838b5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:51:39.069150 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:39.070958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:39.131430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f97ea-41d7-47c2-8385-77ddc85ef31d None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:39.181409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:39.242227 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21415f85-1076-471d-a705-a6b33f4f4662 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:39.250880 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086951a6-275a-416a-a576-7ec354abf30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:39.251654 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086951a6-275a-416a-a576-7ec354abf30f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:39.252592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086951a6-275a-416a-a576-7ec354abf30f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:39.253019 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086951a6-275a-416a-a576-7ec354abf30f admin admin] PATH_INFO: `/v3/users/87941fe58ba94c0a8715c4f6eb7f35c5` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:39.254332 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086951a6-275a-416a-a576-7ec354abf30f admin admin] RBAC: Authorizing `identity:delete_user(user_id=87941fe58ba94c0a8715c4f6eb7f35c5)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:39.257840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086951a6-275a-416a-a576-7ec354abf30f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:39.259183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502de238-46ab-4bbb-a6f1-246b390e397e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:39.260439 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502de238-46ab-4bbb-a6f1-246b390e397e service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:39.260827 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502de238-46ab-4bbb-a6f1-246b390e397e service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:39.261649 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502de238-46ab-4bbb-a6f1-246b390e397e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:39.264576 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-502de238-46ab-4bbb-a6f1-246b390e397e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:39.293188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:39.382030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-086951a6-275a-416a-a576-7ec354abf30f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87941fe58ba94c0a8715c4f6eb7f35c5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:39.382750 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-086951a6-275a-416a-a576-7ec354abf30f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87941fe58ba94c0a8715c4f6eb7f35c5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:39.383386 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-086951a6-275a-416a-a576-7ec354abf30f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87941fe58ba94c0a8715c4f6eb7f35c5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:39.419978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-086951a6-275a-416a-a576-7ec354abf30f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87941fe58ba94c0a8715c4f6eb7f35c5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:39.448410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502de238-46ab-4bbb-a6f1-246b390e397e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:39.450334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502de238-46ab-4bbb-a6f1-246b390e397e service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:39.458999 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-086951a6-275a-416a-a576-7ec354abf30f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87941fe58ba94c0a8715c4f6eb7f35c5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:39.466835 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 972/1925] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:39.478535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 954/1926] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:39 2022] DELETE /identity/v3/users/87941fe58ba94c0a8715c4f6eb7f35c5 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:40.169819 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502de238-46ab-4bbb-a6f1-246b390e397e service nova] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:40.224721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502de238-46ab-4bbb-a6f1-246b390e397e service nova] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:40.389314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:40.390482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:40.391024 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:40.391443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] PATH_INFO: `/v3/projects/112fb74ca1ec45e6bff1bf23351d4779` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:40.396908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=112fb74ca1ec45e6bff1bf23351d4779)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:40.396908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:40.571594 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '112fb74ca1ec45e6bff1bf23351d4779', 'request_id': 'req-5ad10446-ad54-42fc-8518-3a3be369a4b9'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:40.606974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '112fb74ca1ec45e6bff1bf23351d4779', 'request_id': 'req-5ad10446-ad54-42fc-8518-3a3be369a4b9'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:40.637645 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '112fb74ca1ec45e6bff1bf23351d4779', 'request_id': 'req-5ad10446-ad54-42fc-8518-3a3be369a4b9'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:40.679162 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] The token cache is being invalidate because project 112fb74ca1ec45e6bff1bf23351d4779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:40.679685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:40.684734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:40.695953 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 973/1927] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:40 2022] DELETE /identity/v3/projects/112fb74ca1ec45e6bff1bf23351d4779 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:40.702820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570af2e-5f35-44e9-a1a9-81e2eb0a39b6 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:40.704633 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570af2e-5f35-44e9-a1a9-81e2eb0a39b6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:40.705333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570af2e-5f35-44e9-a1a9-81e2eb0a39b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:40.773813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e570af2e-5f35-44e9-a1a9-81e2eb0a39b6 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:40.777957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e570af2e-5f35-44e9-a1a9-81e2eb0a39b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:40.894179 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:40.903165 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:41.078655 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 955/1928] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:51:41.084845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3b5f3-e2f4-48e3-b2e3-036a9eb47530 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:41.087285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3b5f3-e2f4-48e3-b2e3-036a9eb47530 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:41.087740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3b5f3-e2f4-48e3-b2e3-036a9eb47530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:41.102797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:41.135393 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-69d3b5f3-e2f4-48e3-b2e3-036a9eb47530 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:41.137516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-69d3b5f3-e2f4-48e3-b2e3-036a9eb47530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:41.143080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad10446-ad54-42fc-8518-3a3be369a4b9 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:41.154644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d3de4-9347-4652-9720-34c579caa41b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6720518c03413aa8f85e4d6142fe87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:41.154644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d3de4-9347-4652-9720-34c579caa41b service placement] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:41.154644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d3de4-9347-4652-9720-34c579caa41b service placement] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:41.154644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d3de4-9347-4652-9720-34c579caa41b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:41.157257 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-250d3de4-9347-4652-9720-34c579caa41b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:41.161729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 956/1929] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:41.163013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3b5f3-e2f4-48e3-b2e3-036a9eb47530 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:41.174435 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3b5f3-e2f4-48e3-b2e3-036a9eb47530 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:41.176053 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d3de4-9347-4652-9720-34c579caa41b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:41.177816 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d3de4-9347-4652-9720-34c579caa41b service placement] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:41.181486 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5a8ef-4aaa-4e38-baee-6e982332d8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:41.184597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5a8ef-4aaa-4e38-baee-6e982332d8fd admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:41.186327 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5a8ef-4aaa-4e38-baee-6e982332d8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:41.186886 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5a8ef-4aaa-4e38-baee-6e982332d8fd admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:41.190644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5a8ef-4aaa-4e38-baee-6e982332d8fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:51:41.191100 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5a8ef-4aaa-4e38-baee-6e982332d8fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:41.196843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5a8ef-4aaa-4e38-baee-6e982332d8fd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:41.198812 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 974/1930] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:41.220665 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 957/1931] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:51:41.229031 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d3de4-9347-4652-9720-34c579caa41b service placement] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:41.247473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d3de4-9347-4652-9720-34c579caa41b service placement] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:41.254382 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb8d7d-725f-42b6-bd4a-5b2db718623d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:41.255173 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb8d7d-725f-42b6-bd4a-5b2db718623d admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:41.255550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb8d7d-725f-42b6-bd4a-5b2db718623d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:41.255956 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb8d7d-725f-42b6-bd4a-5b2db718623d admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:41.256719 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb8d7d-725f-42b6-bd4a-5b2db718623d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:41.259096 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb8d7d-725f-42b6-bd4a-5b2db718623d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:41.329127 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5a8ef-4aaa-4e38-baee-6e982332d8fd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:41.341191 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 975/1932] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:51:41 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:41.345423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5a8ef-4aaa-4e38-baee-6e982332d8fd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:41.412670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:41.450666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24aef4-69d1-4c97-9064-8500ed3c6d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:41.451450 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24aef4-69d1-4c97-9064-8500ed3c6d08 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:41.451825 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24aef4-69d1-4c97-9064-8500ed3c6d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:41.452367 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24aef4-69d1-4c97-9064-8500ed3c6d08 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:41.453379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24aef4-69d1-4c97-9064-8500ed3c6d08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:41.455694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24aef4-69d1-4c97-9064-8500ed3c6d08 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:41.498452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 958/1933] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:51:41 2022] POST /identity/v3/users => generated 291 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:41.503215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb8d7d-725f-42b6-bd4a-5b2db718623d admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:41.523010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb8d7d-725f-42b6-bd4a-5b2db718623d admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:41.532006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d82e63-5777-4ed8-bdb8-9b05c5d67eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:41.533593 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d82e63-5777-4ed8-bdb8-9b05c5d67eba admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:41.534416 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d82e63-5777-4ed8-bdb8-9b05c5d67eba admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:41.535021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d82e63-5777-4ed8-bdb8-9b05c5d67eba admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:41.536228 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d82e63-5777-4ed8-bdb8-9b05c5d67eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:41.538703 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d82e63-5777-4ed8-bdb8-9b05c5d67eba admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:41.566085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 976/1934] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:41.571611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24aef4-69d1-4c97-9064-8500ed3c6d08 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:41.588803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24aef4-69d1-4c97-9064-8500ed3c6d08 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:41.604740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafbb064-706f-4644-9543-4755bb3a787d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:41.605668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbb064-706f-4644-9543-4755bb3a787d admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:41.607084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbb064-706f-4644-9543-4755bb3a787d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:41.607545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbb064-706f-4644-9543-4755bb3a787d admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:41.609700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafbb064-706f-4644-9543-4755bb3a787d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:41.612271 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafbb064-706f-4644-9543-4755bb3a787d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:41.684682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 959/1935] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:51:41.690709 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d82e63-5777-4ed8-bdb8-9b05c5d67eba admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:41.710271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d82e63-5777-4ed8-bdb8-9b05c5d67eba admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:41.722017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87627668-5922-49fb-b2a9-342f27f17837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:41.722965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87627668-5922-49fb-b2a9-342f27f17837 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:41.723328 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87627668-5922-49fb-b2a9-342f27f17837 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:41.723612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87627668-5922-49fb-b2a9-342f27f17837 admin admin] PATH_INFO: `/v3/projects/44ec1225085e4e5388ff7269db68b7bd/users/32917831f13642e2a7ef99aaab151532/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:41.775357 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87627668-5922-49fb-b2a9-342f27f17837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44ec1225085e4e5388ff7269db68b7bd, user_id=32917831f13642e2a7ef99aaab151532, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:41.778469 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87627668-5922-49fb-b2a9-342f27f17837 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:41.806106 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 977/1936] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:41 2022] PUT /identity/v3/projects/44ec1225085e4e5388ff7269db68b7bd/users/32917831f13642e2a7ef99aaab151532/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:41.824323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae1107-751a-47ed-a33f-c114820c414a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:41.825016 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae1107-751a-47ed-a33f-c114820c414a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:41.825488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae1107-751a-47ed-a33f-c114820c414a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:41.895409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f0ae1107-751a-47ed-a33f-c114820c414a None None] MFA Rules not processed for user `32917831f13642e2a7ef99aaab151532`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:41.897098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f0ae1107-751a-47ed-a33f-c114820c414a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:42.027794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 960/1937] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:42.032822 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87627668-5922-49fb-b2a9-342f27f17837 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:42.208656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87627668-5922-49fb-b2a9-342f27f17837 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:42.227776 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ce1bc-9cd9-4ed6-b222-368553cdffe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:42.231609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ce1bc-9cd9-4ed6-b222-368553cdffe0 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:42.232652 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ce1bc-9cd9-4ed6-b222-368553cdffe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:42.233054 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ce1bc-9cd9-4ed6-b222-368553cdffe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:42.234504 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ce1bc-9cd9-4ed6-b222-368553cdffe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:42.240508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ce1bc-9cd9-4ed6-b222-368553cdffe0 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:42.291940 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 978/1938] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:51:42 2022] POST /identity/v3/projects => generated 383 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:42.295683 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae1107-751a-47ed-a33f-c114820c414a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:42.311311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae1107-751a-47ed-a33f-c114820c414a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:42.477605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3642a-4cdf-4999-bb5f-6b50ca77abba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:42.479307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3642a-4cdf-4999-bb5f-6b50ca77abba admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:42.479608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3642a-4cdf-4999-bb5f-6b50ca77abba admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:42.479868 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3642a-4cdf-4999-bb5f-6b50ca77abba admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:42.481478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3642a-4cdf-4999-bb5f-6b50ca77abba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:42.483951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3642a-4cdf-4999-bb5f-6b50ca77abba admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:42.524680 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8217c67-95d3-4b1d-bac9-fc2c64ea60e1 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:42.528707 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8217c67-95d3-4b1d-bac9-fc2c64ea60e1 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:42.529537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8217c67-95d3-4b1d-bac9-fc2c64ea60e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:42.542962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 961/1939] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:51:42 2022] POST /identity/v3/users => generated 298 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:51:42.546789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3642a-4cdf-4999-bb5f-6b50ca77abba admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:42.579297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3642a-4cdf-4999-bb5f-6b50ca77abba admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:42.601003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-a8217c67-95d3-4b1d-bac9-fc2c64ea60e1 None None] MFA Rules not processed for user `ed6e530c3f544748bd3ffc256b1630b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:42.601341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb37e48-c7ce-4ca0-9f48-60e9ca313786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:42.602229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb37e48-c7ce-4ca0-9f48-60e9ca313786 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:42.602653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb37e48-c7ce-4ca0-9f48-60e9ca313786 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:42.603428 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb37e48-c7ce-4ca0-9f48-60e9ca313786 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:42.605107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb37e48-c7ce-4ca0-9f48-60e9ca313786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:42.606642 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-a8217c67-95d3-4b1d-bac9-fc2c64ea60e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:42.608671 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb37e48-c7ce-4ca0-9f48-60e9ca313786 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:42.636038 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 962/1940] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:42.643952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb37e48-c7ce-4ca0-9f48-60e9ca313786 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:42.678822 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb37e48-c7ce-4ca0-9f48-60e9ca313786 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:42.685558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa806b69-d4ff-46d0-aee0-119a9ebbdd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:42.686723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa806b69-d4ff-46d0-aee0-119a9ebbdd39 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:42.687492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa806b69-d4ff-46d0-aee0-119a9ebbdd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:42.688116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa806b69-d4ff-46d0-aee0-119a9ebbdd39 admin admin] PATH_INFO: `/v3/projects/1f0aedce6ee04d5aa342a2695fa78431/users/3c541f5175b640748f933fef2410dfee/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:42.728504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa806b69-d4ff-46d0-aee0-119a9ebbdd39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f0aedce6ee04d5aa342a2695fa78431, user_id=3c541f5175b640748f933fef2410dfee, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:42.732394 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa806b69-d4ff-46d0-aee0-119a9ebbdd39 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:42.737522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 979/1941] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:42.770773 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 963/1942] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:42 2022] PUT /identity/v3/projects/1f0aedce6ee04d5aa342a2695fa78431/users/3c541f5175b640748f933fef2410dfee/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:42.772674 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8217c67-95d3-4b1d-bac9-fc2c64ea60e1 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:42.791109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8217c67-95d3-4b1d-bac9-fc2c64ea60e1 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:42.808456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:42.953352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbdd06-83aa-4c14-be5b-02022a24b142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:42.954520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbdd06-83aa-4c14-be5b-02022a24b142 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:42.955224 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbdd06-83aa-4c14-be5b-02022a24b142 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:42.955996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbdd06-83aa-4c14-be5b-02022a24b142 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:42.957518 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbdd06-83aa-4c14-be5b-02022a24b142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:42.963033 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbdd06-83aa-4c14-be5b-02022a24b142 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:42.983969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 980/1943] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:42.989064 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa806b69-d4ff-46d0-aee0-119a9ebbdd39 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:43.002528 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:43.022814 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa806b69-d4ff-46d0-aee0-119a9ebbdd39 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:43.033789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97c02d-8954-4855-9b56-a91b02d21a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:43.035104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97c02d-8954-4855-9b56-a91b02d21a6e admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:43.035603 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97c02d-8954-4855-9b56-a91b02d21a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:43.036116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97c02d-8954-4855-9b56-a91b02d21a6e admin admin] PATH_INFO: `/v3/projects/1f0aedce6ee04d5aa342a2695fa78431/users/3c541f5175b640748f933fef2410dfee/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:43.038623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97c02d-8954-4855-9b56-a91b02d21a6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f0aedce6ee04d5aa342a2695fa78431, user_id=3c541f5175b640748f933fef2410dfee, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:43.041712 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97c02d-8954-4855-9b56-a91b02d21a6e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:43.057758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 964/1944] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:42 2022] PUT /identity/v3/projects/1f0aedce6ee04d5aa342a2695fa78431/users/3c541f5175b640748f933fef2410dfee/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:43.075566 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58527-40df-498f-a7a6-0bc65b4c3ad0 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:43.075911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58527-40df-498f-a7a6-0bc65b4c3ad0 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:43.076853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58527-40df-498f-a7a6-0bc65b4c3ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:43.151337 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-16b58527-40df-498f-a7a6-0bc65b4c3ad0 None None] MFA Rules not processed for user `3c541f5175b640748f933fef2410dfee`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:43.153174 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-16b58527-40df-498f-a7a6-0bc65b4c3ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:43.329149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 981/1945] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:43.345745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97c02d-8954-4855-9b56-a91b02d21a6e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:43.348680 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5d97c02d-8954-4855-9b56-a91b02d21a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:43.489815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97c02d-8954-4855-9b56-a91b02d21a6e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:43.497537 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6ddf4-de51-4e0d-8107-f0f38c53a911 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:43.498582 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6ddf4-de51-4e0d-8107-f0f38c53a911 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:43.499029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6ddf4-de51-4e0d-8107-f0f38c53a911 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:43.500714 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6ddf4-de51-4e0d-8107-f0f38c53a911 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:43.519131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6ddf4-de51-4e0d-8107-f0f38c53a911 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:43.521302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6ddf4-de51-4e0d-8107-f0f38c53a911 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:43.541281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 965/1946] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:43.960691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b58527-40df-498f-a7a6-0bc65b4c3ad0 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:43.976289 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b58527-40df-498f-a7a6-0bc65b4c3ad0 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:43.988644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e20591-01b6-49fa-b4d6-e137800dacb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:43.989766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e20591-01b6-49fa-b4d6-e137800dacb3 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:43.990280 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e20591-01b6-49fa-b4d6-e137800dacb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:43.990660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e20591-01b6-49fa-b4d6-e137800dacb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:44.133338 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e20591-01b6-49fa-b4d6-e137800dacb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:44.135577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e20591-01b6-49fa-b4d6-e137800dacb3 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:44.183520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 982/1947] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:48.031822 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6ddf4-de51-4e0d-8107-f0f38c53a911 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:48.151089 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6ddf4-de51-4e0d-8107-f0f38c53a911 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:48.158626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c68eb-73a7-4c9a-a62c-7a29434f3be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:48.160073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c68eb-73a7-4c9a-a62c-7a29434f3be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:48.160535 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c68eb-73a7-4c9a-a62c-7a29434f3be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:48.160946 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c68eb-73a7-4c9a-a62c-7a29434f3be2 admin admin] PATH_INFO: `/v3/users/32917831f13642e2a7ef99aaab151532` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:48.163185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c68eb-73a7-4c9a-a62c-7a29434f3be2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32917831f13642e2a7ef99aaab151532)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:48.166875 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c68eb-73a7-4c9a-a62c-7a29434f3be2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:48.258694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f8c68eb-73a7-4c9a-a62c-7a29434f3be2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32917831f13642e2a7ef99aaab151532'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:48.259456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f8c68eb-73a7-4c9a-a62c-7a29434f3be2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32917831f13642e2a7ef99aaab151532'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:48.260707 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f8c68eb-73a7-4c9a-a62c-7a29434f3be2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32917831f13642e2a7ef99aaab151532'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:48.277727 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f8c68eb-73a7-4c9a-a62c-7a29434f3be2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32917831f13642e2a7ef99aaab151532'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:48.290514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f8c68eb-73a7-4c9a-a62c-7a29434f3be2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32917831f13642e2a7ef99aaab151532'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:48.322350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 966/1948] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:48 2022] DELETE /identity/v3/users/32917831f13642e2a7ef99aaab151532 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:48.329288 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d669576-9ef2-4d59-8c03-fbca711a8ff9 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:48.329950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d669576-9ef2-4d59-8c03-fbca711a8ff9 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:48.329950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d669576-9ef2-4d59-8c03-fbca711a8ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:48.370940 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-1d669576-9ef2-4d59-8c03-fbca711a8ff9 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:48.377153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1d669576-9ef2-4d59-8c03-fbca711a8ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:48.454559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:48.555755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 983/1949] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:48.563658 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c68eb-73a7-4c9a-a62c-7a29434f3be2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:48.578964 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9f8c68eb-73a7-4c9a-a62c-7a29434f3be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:48.628798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:48.737139 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c68eb-73a7-4c9a-a62c-7a29434f3be2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:48.744431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beff728-0369-4b66-8af2-8135652efc5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:48.746008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beff728-0369-4b66-8af2-8135652efc5b service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:48.746775 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beff728-0369-4b66-8af2-8135652efc5b service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:48.747514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beff728-0369-4b66-8af2-8135652efc5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:48.797779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beff728-0369-4b66-8af2-8135652efc5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:48.801089 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beff728-0369-4b66-8af2-8135652efc5b service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:48.854699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 967/1950] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:49.727760 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d669576-9ef2-4d59-8c03-fbca711a8ff9 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:49.734836 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1d669576-9ef2-4d59-8c03-fbca711a8ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:49.762253 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d669576-9ef2-4d59-8c03-fbca711a8ff9 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:49.772848 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:49.774075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:49.774517 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:49.774891 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] PATH_INFO: `/v3/projects/44ec1225085e4e5388ff7269db68b7bd` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:49.778527 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44ec1225085e4e5388ff7269db68b7bd)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:49.782617 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:49.907397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44ec1225085e4e5388ff7269db68b7bd', 'request_id': 'req-4defa880-f58d-4e27-a747-4a8a8ccf68a4'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:49.920932 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44ec1225085e4e5388ff7269db68b7bd', 'request_id': 'req-4defa880-f58d-4e27-a747-4a8a8ccf68a4'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:49.930686 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44ec1225085e4e5388ff7269db68b7bd', 'request_id': 'req-4defa880-f58d-4e27-a747-4a8a8ccf68a4'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:49.948089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] The token cache is being invalidate because project 44ec1225085e4e5388ff7269db68b7bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:49.948732 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:49.949659 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:49.951725 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 984/1951] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:49 2022] DELETE /identity/v3/projects/44ec1225085e4e5388ff7269db68b7bd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:49.958187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beff728-0369-4b66-8af2-8135652efc5b service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:49.982666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beff728-0369-4b66-8af2-8135652efc5b service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:50.131209 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6915f-58ce-467a-8cec-9ec1c2112306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:50.133086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6915f-58ce-467a-8cec-9ec1c2112306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:50.133709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6915f-58ce-467a-8cec-9ec1c2112306 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:50.134487 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6915f-58ce-467a-8cec-9ec1c2112306 admin admin] PATH_INFO: `/v3/users/3c541f5175b640748f933fef2410dfee` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:50.136898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6915f-58ce-467a-8cec-9ec1c2112306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c541f5175b640748f933fef2410dfee)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:50.142078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6915f-58ce-467a-8cec-9ec1c2112306 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:50.370288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c0a6915f-58ce-467a-8cec-9ec1c2112306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c541f5175b640748f933fef2410dfee'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:50.372772 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c0a6915f-58ce-467a-8cec-9ec1c2112306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c541f5175b640748f933fef2410dfee'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:50.374981 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c0a6915f-58ce-467a-8cec-9ec1c2112306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c541f5175b640748f933fef2410dfee'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:50.418878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c0a6915f-58ce-467a-8cec-9ec1c2112306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c541f5175b640748f933fef2410dfee'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:50.458838 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c0a6915f-58ce-467a-8cec-9ec1c2112306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c541f5175b640748f933fef2410dfee'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:50.514676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 968/1952] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:49 2022] DELETE /identity/v3/users/3c541f5175b640748f933fef2410dfee => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:50.954470 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:50.957091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:50.978506 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defa880-f58d-4e27-a747-4a8a8ccf68a4 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:51.133087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e1444-7daa-4c4f-a377-259af1ee1d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:51.134662 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e1444-7daa-4c4f-a377-259af1ee1d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:51.135342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e1444-7daa-4c4f-a377-259af1ee1d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:51.136034 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e1444-7daa-4c4f-a377-259af1ee1d56 admin admin] PATH_INFO: `/v3/projects/1f0aedce6ee04d5aa342a2695fa78431` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:51.139530 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e1444-7daa-4c4f-a377-259af1ee1d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f0aedce6ee04d5aa342a2695fa78431)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:51.143532 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e1444-7daa-4c4f-a377-259af1ee1d56 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:51.286492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ba7e1444-7daa-4c4f-a377-259af1ee1d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f0aedce6ee04d5aa342a2695fa78431', 'request_id': 'req-ba7e1444-7daa-4c4f-a377-259af1ee1d56'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:51.326825 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ba7e1444-7daa-4c4f-a377-259af1ee1d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f0aedce6ee04d5aa342a2695fa78431', 'request_id': 'req-ba7e1444-7daa-4c4f-a377-259af1ee1d56'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:51.338890 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:51.350240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ba7e1444-7daa-4c4f-a377-259af1ee1d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f0aedce6ee04d5aa342a2695fa78431', 'request_id': 'req-ba7e1444-7daa-4c4f-a377-259af1ee1d56'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:51.408004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ba7e1444-7daa-4c4f-a377-259af1ee1d56 admin admin] The token cache is being invalidate because project 1f0aedce6ee04d5aa342a2695fa78431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:51.408004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ba7e1444-7daa-4c4f-a377-259af1ee1d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:51.408004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ba7e1444-7daa-4c4f-a377-259af1ee1d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:51.408004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 985/1953] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:50 2022] DELETE /identity/v3/projects/1f0aedce6ee04d5aa342a2695fa78431 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:51.414930 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3bc1e-de89-4a48-96ca-ded2a20d6216 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:51.415651 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3bc1e-de89-4a48-96ca-ded2a20d6216 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:51.416028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3bc1e-de89-4a48-96ca-ded2a20d6216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:51.510675 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0bc3bc1e-de89-4a48-96ca-ded2a20d6216 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:51.510675 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0bc3bc1e-de89-4a48-96ca-ded2a20d6216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:51.674369 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:51.822813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 969/1954] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:51.825586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560535c3-94ba-4ccf-bfad-434b0cdde9ff None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:51.826358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560535c3-94ba-4ccf-bfad-434b0cdde9ff None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:51.828924 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560535c3-94ba-4ccf-bfad-434b0cdde9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:51.914947 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-560535c3-94ba-4ccf-bfad-434b0cdde9ff None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:51.917541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-560535c3-94ba-4ccf-bfad-434b0cdde9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:51.937986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3bc1e-de89-4a48-96ca-ded2a20d6216 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:51.940738 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 986/1955] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:51:51.959622 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3bc1e-de89-4a48-96ca-ded2a20d6216 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:51.971058 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eae38-e85e-4445-93cc-dce73fb508ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:51.972340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eae38-e85e-4445-93cc-dce73fb508ff admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:51.972948 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eae38-e85e-4445-93cc-dce73fb508ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:51.973534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eae38-e85e-4445-93cc-dce73fb508ff admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:51.974859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23eae38-e85e-4445-93cc-dce73fb508ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:51:51.975521 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23eae38-e85e-4445-93cc-dce73fb508ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:51.979468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23eae38-e85e-4445-93cc-dce73fb508ff admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:52.015128 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 970/1956] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:51:52.020425 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560535c3-94ba-4ccf-bfad-434b0cdde9ff None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:52.030630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560535c3-94ba-4ccf-bfad-434b0cdde9ff None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:52.070698 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4c555-b949-4374-8b9c-a45652f85be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:52.071553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4c555-b949-4374-8b9c-a45652f85be5 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:52.073072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4c555-b949-4374-8b9c-a45652f85be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:52.073072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4c555-b949-4374-8b9c-a45652f85be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:52.076985 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4c555-b949-4374-8b9c-a45652f85be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:52.080530 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4c555-b949-4374-8b9c-a45652f85be5 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:52.203517 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 987/1957] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:51:52 2022] POST /identity/v3/projects => generated 381 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:52.205541 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eae38-e85e-4445-93cc-dce73fb508ff admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:52.238034 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eae38-e85e-4445-93cc-dce73fb508ff admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:52.385551 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5179e14-fe8c-4e7b-8008-7a8ba45f2c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:52.388616 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5179e14-fe8c-4e7b-8008-7a8ba45f2c8a admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:52.388984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5179e14-fe8c-4e7b-8008-7a8ba45f2c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:52.389331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5179e14-fe8c-4e7b-8008-7a8ba45f2c8a admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:52.391085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5179e14-fe8c-4e7b-8008-7a8ba45f2c8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:52.394325 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5179e14-fe8c-4e7b-8008-7a8ba45f2c8a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:52.427220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 971/1958] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:51:52 2022] POST /identity/v3/users => generated 291 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:51:52.432895 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4c555-b949-4374-8b9c-a45652f85be5 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:52.460044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4c555-b949-4374-8b9c-a45652f85be5 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:52.468853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287155d-551c-47ef-9dbe-1a31ff71c40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:52.471452 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287155d-551c-47ef-9dbe-1a31ff71c40c admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:52.471741 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287155d-551c-47ef-9dbe-1a31ff71c40c admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:52.472058 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287155d-551c-47ef-9dbe-1a31ff71c40c admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:52.472735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287155d-551c-47ef-9dbe-1a31ff71c40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:52.474912 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287155d-551c-47ef-9dbe-1a31ff71c40c admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:52.493602 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 988/1959] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:52.497967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5179e14-fe8c-4e7b-8008-7a8ba45f2c8a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:52.509783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5179e14-fe8c-4e7b-8008-7a8ba45f2c8a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:52.516830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b94556-2c1a-4e1a-b037-54b15b3be4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:52.517841 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b94556-2c1a-4e1a-b037-54b15b3be4ec admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:52.518420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b94556-2c1a-4e1a-b037-54b15b3be4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:52.518902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b94556-2c1a-4e1a-b037-54b15b3be4ec admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:52.519781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b94556-2c1a-4e1a-b037-54b15b3be4ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:52.521894 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b94556-2c1a-4e1a-b037-54b15b3be4ec admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:52.537878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 972/1960] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:52.542001 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287155d-551c-47ef-9dbe-1a31ff71c40c admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:52.553267 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287155d-551c-47ef-9dbe-1a31ff71c40c admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:52.558121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c22fd-171b-451f-a2c4-b84cccc27e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:52.558806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c22fd-171b-451f-a2c4-b84cccc27e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:52.559078 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c22fd-171b-451f-a2c4-b84cccc27e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:52.559343 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c22fd-171b-451f-a2c4-b84cccc27e8d admin admin] PATH_INFO: `/v3/projects/c81b45cdf17648be8c9830541fb48892/users/321c5ffe0a8d45f08f1394ab117b210a/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:52.578509 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c22fd-171b-451f-a2c4-b84cccc27e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c81b45cdf17648be8c9830541fb48892, user_id=321c5ffe0a8d45f08f1394ab117b210a, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:52.581265 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c22fd-171b-451f-a2c4-b84cccc27e8d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:52.604168 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 989/1961] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:52 2022] PUT /identity/v3/projects/c81b45cdf17648be8c9830541fb48892/users/321c5ffe0a8d45f08f1394ab117b210a/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:52.612657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e54f38-23e5-48b9-a915-4a8e8a57fe98 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:52.612987 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e54f38-23e5-48b9-a915-4a8e8a57fe98 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:52.613231 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e54f38-23e5-48b9-a915-4a8e8a57fe98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:52.647789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a0e54f38-23e5-48b9-a915-4a8e8a57fe98 None None] MFA Rules not processed for user `321c5ffe0a8d45f08f1394ab117b210a`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:52.649532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a0e54f38-23e5-48b9-a915-4a8e8a57fe98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:52.755290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 973/1962] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:52.759938 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c22fd-171b-451f-a2c4-b84cccc27e8d admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:52.851175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c22fd-171b-451f-a2c4-b84cccc27e8d admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:52.862286 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ca2b0-6b32-4b46-9927-752f723d45e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:52.862620 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ca2b0-6b32-4b46-9927-752f723d45e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:52.862909 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ca2b0-6b32-4b46-9927-752f723d45e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:52.863215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ca2b0-6b32-4b46-9927-752f723d45e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:52.864072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ca2b0-6b32-4b46-9927-752f723d45e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:52.866540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ca2b0-6b32-4b46-9927-752f723d45e5 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:52.947279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 990/1963] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:51:52 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:51:52.953630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e54f38-23e5-48b9-a915-4a8e8a57fe98 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:52.972221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e54f38-23e5-48b9-a915-4a8e8a57fe98 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:53.026470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:53.081751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3d45c-8f5c-43f0-8259-1d55abebf757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:53.082635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3d45c-8f5c-43f0-8259-1d55abebf757 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:53.083032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3d45c-8f5c-43f0-8259-1d55abebf757 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:53.083428 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3d45c-8f5c-43f0-8259-1d55abebf757 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:53.084433 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3d45c-8f5c-43f0-8259-1d55abebf757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:53.086898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3d45c-8f5c-43f0-8259-1d55abebf757 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:53.120688 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 974/1964] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:51:52 2022] POST /identity/v3/users => generated 294 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:53.125484 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ca2b0-6b32-4b46-9927-752f723d45e5 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:53.138843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ca2b0-6b32-4b46-9927-752f723d45e5 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:53.144906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d1769-2d63-4b9f-ad7b-9aa380e4552f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:53.145613 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d1769-2d63-4b9f-ad7b-9aa380e4552f admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:53.145948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d1769-2d63-4b9f-ad7b-9aa380e4552f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:53.146280 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d1769-2d63-4b9f-ad7b-9aa380e4552f admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:53.147072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d1769-2d63-4b9f-ad7b-9aa380e4552f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:53.148945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d1769-2d63-4b9f-ad7b-9aa380e4552f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:53.169369 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 991/1965] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:53.173794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3d45c-8f5c-43f0-8259-1d55abebf757 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:53.186222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3d45c-8f5c-43f0-8259-1d55abebf757 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:53.193053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240315ee-089e-4600-84fa-6aa1f6f8e2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:53.194183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240315ee-089e-4600-84fa-6aa1f6f8e2eb admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:53.194183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240315ee-089e-4600-84fa-6aa1f6f8e2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:53.194615 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240315ee-089e-4600-84fa-6aa1f6f8e2eb admin admin] PATH_INFO: `/v3/projects/24c3fb3198f540e38a549d9353092510/users/f51b67205a7f4f1c952a6d1f06d61f75/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:53.221861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240315ee-089e-4600-84fa-6aa1f6f8e2eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c3fb3198f540e38a549d9353092510, user_id=f51b67205a7f4f1c952a6d1f06d61f75, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:53.227438 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240315ee-089e-4600-84fa-6aa1f6f8e2eb admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:53.258940 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 975/1966] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:53 2022] PUT /identity/v3/projects/24c3fb3198f540e38a549d9353092510/users/f51b67205a7f4f1c952a6d1f06d61f75/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:53.266590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d1769-2d63-4b9f-ad7b-9aa380e4552f admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:53.282540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d1769-2d63-4b9f-ad7b-9aa380e4552f admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:53.434311 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:53.445378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445d516-d1e0-4f0a-9921-d43334def0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:53.447507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445d516-d1e0-4f0a-9921-d43334def0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:53.448749 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445d516-d1e0-4f0a-9921-d43334def0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:53.449349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445d516-d1e0-4f0a-9921-d43334def0c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:53.451671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445d516-d1e0-4f0a-9921-d43334def0c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:53.454070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445d516-d1e0-4f0a-9921-d43334def0c8 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:53.473522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 992/1967] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:53.474915 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240315ee-089e-4600-84fa-6aa1f6f8e2eb admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:53.498564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240315ee-089e-4600-84fa-6aa1f6f8e2eb admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:53.507145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5df6e-18b6-4ce1-8a8e-f4052f34a036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:53.508664 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5df6e-18b6-4ce1-8a8e-f4052f34a036 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:53.509221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5df6e-18b6-4ce1-8a8e-f4052f34a036 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:53.509936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5df6e-18b6-4ce1-8a8e-f4052f34a036 admin admin] PATH_INFO: `/v3/projects/24c3fb3198f540e38a549d9353092510/users/f51b67205a7f4f1c952a6d1f06d61f75/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:53.520953 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5df6e-18b6-4ce1-8a8e-f4052f34a036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c3fb3198f540e38a549d9353092510, user_id=f51b67205a7f4f1c952a6d1f06d61f75, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:53.525215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5df6e-18b6-4ce1-8a8e-f4052f34a036 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:53.556954 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 976/1968] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:53 2022] PUT /identity/v3/projects/24c3fb3198f540e38a549d9353092510/users/f51b67205a7f4f1c952a6d1f06d61f75/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:51:53.559712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44493252-358c-40ae-a9c6-1f6ced8a39f7 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:53.559836 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44493252-358c-40ae-a9c6-1f6ced8a39f7 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:53.560116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44493252-358c-40ae-a9c6-1f6ced8a39f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:53.659655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-44493252-358c-40ae-a9c6-1f6ced8a39f7 None None] MFA Rules not processed for user `f51b67205a7f4f1c952a6d1f06d61f75`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:53.661965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-44493252-358c-40ae-a9c6-1f6ced8a39f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:53.780003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 993/1969] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:51:53.792379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5df6e-18b6-4ce1-8a8e-f4052f34a036 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:53.798662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-faf5df6e-18b6-4ce1-8a8e-f4052f34a036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:53.874540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5df6e-18b6-4ce1-8a8e-f4052f34a036 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:53.878397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a5b13-215f-4559-be89-b67a4515a9e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:53.879143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a5b13-215f-4559-be89-b67a4515a9e6 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:53.879516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a5b13-215f-4559-be89-b67a4515a9e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:53.879865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a5b13-215f-4559-be89-b67a4515a9e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:53.894951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a5b13-215f-4559-be89-b67a4515a9e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:53.896722 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a5b13-215f-4559-be89-b67a4515a9e6 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:53.923067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 977/1970] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:54.560655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44493252-358c-40ae-a9c6-1f6ced8a39f7 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:54.761818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44493252-358c-40ae-a9c6-1f6ced8a39f7 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:54.771471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193b824-c6e2-4975-bdb6-f0b494d724e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:54.772386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193b824-c6e2-4975-bdb6-f0b494d724e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:54.773109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193b824-c6e2-4975-bdb6-f0b494d724e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:54.773527 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193b824-c6e2-4975-bdb6-f0b494d724e7 admin admin] PATH_INFO: `/v3/users/321c5ffe0a8d45f08f1394ab117b210a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:54.775963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193b824-c6e2-4975-bdb6-f0b494d724e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=321c5ffe0a8d45f08f1394ab117b210a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:54.781549 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193b824-c6e2-4975-bdb6-f0b494d724e7 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:54.900795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e193b824-c6e2-4975-bdb6-f0b494d724e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '321c5ffe0a8d45f08f1394ab117b210a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:54.914316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e193b824-c6e2-4975-bdb6-f0b494d724e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '321c5ffe0a8d45f08f1394ab117b210a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:54.915146 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e193b824-c6e2-4975-bdb6-f0b494d724e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '321c5ffe0a8d45f08f1394ab117b210a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:54.915866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e193b824-c6e2-4975-bdb6-f0b494d724e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '321c5ffe0a8d45f08f1394ab117b210a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:54.941144 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e193b824-c6e2-4975-bdb6-f0b494d724e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '321c5ffe0a8d45f08f1394ab117b210a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:54.971102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:54.998309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 994/1971] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:54 2022] DELETE /identity/v3/users/321c5ffe0a8d45f08f1394ab117b210a => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:55.003633 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff173d3b-61e5-44a0-90f0-59f449d8a827 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:55.004319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff173d3b-61e5-44a0-90f0-59f449d8a827 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:55.004319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff173d3b-61e5-44a0-90f0-59f449d8a827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:55.044908 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ff173d3b-61e5-44a0-90f0-59f449d8a827 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:55.047740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ff173d3b-61e5-44a0-90f0-59f449d8a827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:55.173447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 978/1972] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:51:55.186540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193b824-c6e2-4975-bdb6-f0b494d724e7 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:55.188736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e193b824-c6e2-4975-bdb6-f0b494d724e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:55.304539 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193b824-c6e2-4975-bdb6-f0b494d724e7 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:55.310013 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c21eb-125a-4e62-b47d-67e740c2d30f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:55.310751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c21eb-125a-4e62-b47d-67e740c2d30f service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:55.311042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c21eb-125a-4e62-b47d-67e740c2d30f service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:55.311353 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c21eb-125a-4e62-b47d-67e740c2d30f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:55.331522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c21eb-125a-4e62-b47d-67e740c2d30f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:55.333376 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c21eb-125a-4e62-b47d-67e740c2d30f service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:55.355773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 995/1973] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:51:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:51:55.934550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff173d3b-61e5-44a0-90f0-59f449d8a827 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:55.936445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ff173d3b-61e5-44a0-90f0-59f449d8a827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:55.953888 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff173d3b-61e5-44a0-90f0-59f449d8a827 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:55.962113 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:55.963316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:55.963689 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:55.964321 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] PATH_INFO: `/v3/projects/c81b45cdf17648be8c9830541fb48892` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:55.966065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c81b45cdf17648be8c9830541fb48892)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:55.969405 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:56.010185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:56.062978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c81b45cdf17648be8c9830541fb48892', 'request_id': 'req-c47d7472-fef4-4556-8c7d-d62003f997f4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:56.076035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c81b45cdf17648be8c9830541fb48892', 'request_id': 'req-c47d7472-fef4-4556-8c7d-d62003f997f4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:56.096467 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c81b45cdf17648be8c9830541fb48892', 'request_id': 'req-c47d7472-fef4-4556-8c7d-d62003f997f4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:56.113522 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] The token cache is being invalidate because project c81b45cdf17648be8c9830541fb48892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:56.114136 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:56.114885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:56.116588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 979/1974] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:55 2022] DELETE /identity/v3/projects/c81b45cdf17648be8c9830541fb48892 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:56.122677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c21eb-125a-4e62-b47d-67e740c2d30f service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:56.140078 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c21eb-125a-4e62-b47d-67e740c2d30f service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:56.364851 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540adb05-6648-4331-97a5-c142674e5543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:56.365909 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540adb05-6648-4331-97a5-c142674e5543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:56.366428 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540adb05-6648-4331-97a5-c142674e5543 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:56.366857 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540adb05-6648-4331-97a5-c142674e5543 admin admin] PATH_INFO: `/v3/users/f51b67205a7f4f1c952a6d1f06d61f75` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:56.370591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540adb05-6648-4331-97a5-c142674e5543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f51b67205a7f4f1c952a6d1f06d61f75)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:56.373213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540adb05-6648-4331-97a5-c142674e5543 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:56.482474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-540adb05-6648-4331-97a5-c142674e5543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f51b67205a7f4f1c952a6d1f06d61f75'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:56.493135 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-540adb05-6648-4331-97a5-c142674e5543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f51b67205a7f4f1c952a6d1f06d61f75'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:56.494069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-540adb05-6648-4331-97a5-c142674e5543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f51b67205a7f4f1c952a6d1f06d61f75'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:56.499986 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-540adb05-6648-4331-97a5-c142674e5543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f51b67205a7f4f1c952a6d1f06d61f75'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:56.515803 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-540adb05-6648-4331-97a5-c142674e5543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f51b67205a7f4f1c952a6d1f06d61f75'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:56.557017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 996/1975] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:51:56 2022] DELETE /identity/v3/users/f51b67205a7f4f1c952a6d1f06d61f75 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:57.175053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:57.177804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:57.202951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d7472-fef4-4556-8c7d-d62003f997f4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:57.330891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a9044-c875-4a2f-878d-11a1545a6d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:57.332075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a9044-c875-4a2f-878d-11a1545a6d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:57.332329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a9044-c875-4a2f-878d-11a1545a6d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:57.332640 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a9044-c875-4a2f-878d-11a1545a6d82 admin admin] PATH_INFO: `/v3/projects/24c3fb3198f540e38a549d9353092510` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:57.334394 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a9044-c875-4a2f-878d-11a1545a6d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24c3fb3198f540e38a549d9353092510)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:57.337090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a9044-c875-4a2f-878d-11a1545a6d82 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:57.409952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-837a9044-c875-4a2f-878d-11a1545a6d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24c3fb3198f540e38a549d9353092510', 'request_id': 'req-837a9044-c875-4a2f-878d-11a1545a6d82'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:57.421362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-837a9044-c875-4a2f-878d-11a1545a6d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24c3fb3198f540e38a549d9353092510', 'request_id': 'req-837a9044-c875-4a2f-878d-11a1545a6d82'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:57.434422 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-837a9044-c875-4a2f-878d-11a1545a6d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24c3fb3198f540e38a549d9353092510', 'request_id': 'req-837a9044-c875-4a2f-878d-11a1545a6d82'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:57.453037 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-837a9044-c875-4a2f-878d-11a1545a6d82 admin admin] The token cache is being invalidate because project 24c3fb3198f540e38a549d9353092510 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:51:57.453833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-837a9044-c875-4a2f-878d-11a1545a6d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:57.455274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-837a9044-c875-4a2f-878d-11a1545a6d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:51:57.457542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 980/1976] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:51:57 2022] DELETE /identity/v3/projects/24c3fb3198f540e38a549d9353092510 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:51:57.464597 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2656c09-77cc-42c4-944b-640d7b9b9976 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:57.466599 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2656c09-77cc-42c4-944b-640d7b9b9976 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:57.467060 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2656c09-77cc-42c4-944b-640d7b9b9976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:57.509111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-b2656c09-77cc-42c4-944b-640d7b9b9976 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:57.511333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b2656c09-77cc-42c4-944b-640d7b9b9976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:57.553074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:57.647534 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 997/1977] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:57.661909 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b8d27-00f5-481d-a3bf-600e4ba2a006 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:57.661909 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b8d27-00f5-481d-a3bf-600e4ba2a006 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:57.661909 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b8d27-00f5-481d-a3bf-600e4ba2a006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:57.693068 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-db8b8d27-00f5-481d-a3bf-600e4ba2a006 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:57.694481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-db8b8d27-00f5-481d-a3bf-600e4ba2a006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:57.740011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 981/1978] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:51:57.745249 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2656c09-77cc-42c4-944b-640d7b9b9976 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:57.771736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2656c09-77cc-42c4-944b-640d7b9b9976 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:57.779570 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26edf3-b2fc-4d28-8e63-b198b40127cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:57.781141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26edf3-b2fc-4d28-8e63-b198b40127cb admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:57.781520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26edf3-b2fc-4d28-8e63-b198b40127cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:57.781994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26edf3-b2fc-4d28-8e63-b198b40127cb admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:57.783129 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26edf3-b2fc-4d28-8e63-b198b40127cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:51:57.783444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26edf3-b2fc-4d28-8e63-b198b40127cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:57.786259 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26edf3-b2fc-4d28-8e63-b198b40127cb admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:57.815111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 998/1979] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:51:57.820536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b8d27-00f5-481d-a3bf-600e4ba2a006 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:57.862945 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b8d27-00f5-481d-a3bf-600e4ba2a006 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:57.904611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab0088-2d7f-43ce-9bf3-e0e30b9d62b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:57.905708 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab0088-2d7f-43ce-9bf3-e0e30b9d62b8 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:57.906165 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab0088-2d7f-43ce-9bf3-e0e30b9d62b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:57.906792 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab0088-2d7f-43ce-9bf3-e0e30b9d62b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:57.909987 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab0088-2d7f-43ce-9bf3-e0e30b9d62b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:57.916488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab0088-2d7f-43ce-9bf3-e0e30b9d62b8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:57.995744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 982/1980] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:51:57 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:51:58.004152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26edf3-b2fc-4d28-8e63-b198b40127cb admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:58.019421 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26edf3-b2fc-4d28-8e63-b198b40127cb admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:58.143926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c15db-8ec0-4fe9-99c6-6c2850e9dcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:58.145057 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c15db-8ec0-4fe9-99c6-6c2850e9dcd4 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:58.145600 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c15db-8ec0-4fe9-99c6-6c2850e9dcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:58.146108 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c15db-8ec0-4fe9-99c6-6c2850e9dcd4 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:58.147283 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c15db-8ec0-4fe9-99c6-6c2850e9dcd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:58.151085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c15db-8ec0-4fe9-99c6-6c2850e9dcd4 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:58.199226 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 999/1981] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:51:57 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:51:58.201496 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab0088-2d7f-43ce-9bf3-e0e30b9d62b8 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:58.222991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab0088-2d7f-43ce-9bf3-e0e30b9d62b8 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:58.237248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a08be-91fa-4f1c-8f48-4e737bfb969d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:58.238738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a08be-91fa-4f1c-8f48-4e737bfb969d admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:58.239409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a08be-91fa-4f1c-8f48-4e737bfb969d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:58.240196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a08be-91fa-4f1c-8f48-4e737bfb969d admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:58.241625 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a08be-91fa-4f1c-8f48-4e737bfb969d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:58.245646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a08be-91fa-4f1c-8f48-4e737bfb969d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:58.252604 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:51:58.275312 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 983/1982] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:58.281283 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c15db-8ec0-4fe9-99c6-6c2850e9dcd4 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:58.324419 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c15db-8ec0-4fe9-99c6-6c2850e9dcd4 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:58.334222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae2fba-c3f4-4d08-aec3-4f4d8ee3c2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:58.335758 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae2fba-c3f4-4d08-aec3-4f4d8ee3c2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:58.336626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae2fba-c3f4-4d08-aec3-4f4d8ee3c2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:58.337262 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae2fba-c3f4-4d08-aec3-4f4d8ee3c2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:58.342204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae2fba-c3f4-4d08-aec3-4f4d8ee3c2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:58.344548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae2fba-c3f4-4d08-aec3-4f4d8ee3c2d7 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:58.372374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1000/1983] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:51:58.378170 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a08be-91fa-4f1c-8f48-4e737bfb969d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:51:58.410793 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a08be-91fa-4f1c-8f48-4e737bfb969d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:51:58.434443 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33d65d-9db5-4990-b35b-b62ce5657bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:51:58.436364 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33d65d-9db5-4990-b35b-b62ce5657bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:58.437986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33d65d-9db5-4990-b35b-b62ce5657bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:58.440091 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33d65d-9db5-4990-b35b-b62ce5657bb1 admin admin] PATH_INFO: `/v3/projects/4172b607df024042948dc4af055677f8/users/16e40bc5611048a59f3ba09b715985cb/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:58.470361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33d65d-9db5-4990-b35b-b62ce5657bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4172b607df024042948dc4af055677f8, user_id=16e40bc5611048a59f3ba09b715985cb, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:51:58.475435 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33d65d-9db5-4990-b35b-b62ce5657bb1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:51:58.505131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 984/1984] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:51:58 2022] PUT /identity/v3/projects/4172b607df024042948dc4af055677f8/users/16e40bc5611048a59f3ba09b715985cb/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:51:58.522180 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a50d6-d80e-4673-bea1-52503837b0b5 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:51:58.523111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a50d6-d80e-4673-bea1-52503837b0b5 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:51:58.523111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a50d6-d80e-4673-bea1-52503837b0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:51:58.573435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-f81a50d6-d80e-4673-bea1-52503837b0b5 None None] MFA Rules not processed for user `16e40bc5611048a59f3ba09b715985cb`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:51:58.575816 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f81a50d6-d80e-4673-bea1-52503837b0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:51:58.695227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1001/1985] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:51:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:01.562177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33d65d-9db5-4990-b35b-b62ce5657bb1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:01.567449 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fb33d65d-9db5-4990-b35b-b62ce5657bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:01.649851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33d65d-9db5-4990-b35b-b62ce5657bb1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:01.655007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a7239-378d-478f-a5e3-6b18ca0d308e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f5aabcb1414413a8dce7b76ce701582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:01.656092 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a7239-378d-478f-a5e3-6b18ca0d308e service glance] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:01.656382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a7239-378d-478f-a5e3-6b18ca0d308e service glance] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:01.656639 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a7239-378d-478f-a5e3-6b18ca0d308e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:01.666983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a7239-378d-478f-a5e3-6b18ca0d308e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:01.668844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a7239-378d-478f-a5e3-6b18ca0d308e service glance] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:01.679831 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 985/1986] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:52:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:52:02.722840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b9155-878c-4a5d-a0da-f19dc30c27d8 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:02.723623 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b9155-878c-4a5d-a0da-f19dc30c27d8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:02.723800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b9155-878c-4a5d-a0da-f19dc30c27d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:02.759646 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-a68b9155-878c-4a5d-a0da-f19dc30c27d8 None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:02.761730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-a68b9155-878c-4a5d-a0da-f19dc30c27d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:02.779840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:02.855594 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1002/1987] 10.222.0.36 () {58 vars in 967 bytes} [Sat Oct 15 13:52:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:02.877569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a7239-378d-478f-a5e3-6b18ca0d308e service glance] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:02.879917 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c95a7239-378d-478f-a5e3-6b18ca0d308e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:03.000506 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a7239-378d-478f-a5e3-6b18ca0d308e service glance] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:03.009429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffa6d9-ffce-418c-a5bb-eb8e5d6aae72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:03.010086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffa6d9-ffce-418c-a5bb-eb8e5d6aae72 service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:03.010363 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffa6d9-ffce-418c-a5bb-eb8e5d6aae72 service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:03.010635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffa6d9-ffce-418c-a5bb-eb8e5d6aae72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:03.020930 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ffa6d9-ffce-418c-a5bb-eb8e5d6aae72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:03.022573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ffa6d9-ffce-418c-a5bb-eb8e5d6aae72 service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:03.032913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 986/1988] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:52:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:52:05.479668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b9155-878c-4a5d-a0da-f19dc30c27d8 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:05.481900 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-a68b9155-878c-4a5d-a0da-f19dc30c27d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:05.597172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b9155-878c-4a5d-a0da-f19dc30c27d8 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:05.602828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c20583-8041-4abe-b36e-877b9f1c8809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:05.604081 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c20583-8041-4abe-b36e-877b9f1c8809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:05.604522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c20583-8041-4abe-b36e-877b9f1c8809 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:05.605084 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c20583-8041-4abe-b36e-877b9f1c8809 admin admin] PATH_INFO: `/v3/users/af1daddbc1b84ab79f269e124cbea4fb` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:05.606650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c20583-8041-4abe-b36e-877b9f1c8809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1daddbc1b84ab79f269e124cbea4fb)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:05.609238 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c20583-8041-4abe-b36e-877b9f1c8809 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:05.685326 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-d0c20583-8041-4abe-b36e-877b9f1c8809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1daddbc1b84ab79f269e124cbea4fb'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:05.701392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-d0c20583-8041-4abe-b36e-877b9f1c8809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1daddbc1b84ab79f269e124cbea4fb'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:05.713900 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-d0c20583-8041-4abe-b36e-877b9f1c8809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1daddbc1b84ab79f269e124cbea4fb'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:05.713900 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-d0c20583-8041-4abe-b36e-877b9f1c8809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1daddbc1b84ab79f269e124cbea4fb'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:05.721254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-d0c20583-8041-4abe-b36e-877b9f1c8809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1daddbc1b84ab79f269e124cbea4fb'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:05.754239 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1003/1989] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:52:05 2022] DELETE /identity/v3/users/af1daddbc1b84ab79f269e124cbea4fb => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:06.040452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffa6d9-ffce-418c-a5bb-eb8e5d6aae72 service swift] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:06.057125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffa6d9-ffce-418c-a5bb-eb8e5d6aae72 service swift] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:06.145462 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:06.145958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:06.146230 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:06.146493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2 admin admin] PATH_INFO: `/v3/projects/b58d317ad84e46be85ed15b3e51c2f97` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:06.147648 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58d317ad84e46be85ed15b3e51c2f97)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:06.151300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:06.161744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:06.210197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58d317ad84e46be85ed15b3e51c2f97', 'request_id': 'req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:06.217029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58d317ad84e46be85ed15b3e51c2f97', 'request_id': 'req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:06.228904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58d317ad84e46be85ed15b3e51c2f97', 'request_id': 'req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:06.242481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2 admin admin] The token cache is being invalidate because project b58d317ad84e46be85ed15b3e51c2f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:52:06.242746 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:06.243499 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d21d7c34-c6aa-44a2-98c4-4bc38661f6a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:06.245131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 987/1990] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:52:06 2022] DELETE /identity/v3/projects/b58d317ad84e46be85ed15b3e51c2f97 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:06.252378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feb13a-6cad-472b-a59a-0e80733b4d0d None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:06.252754 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feb13a-6cad-472b-a59a-0e80733b4d0d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:06.253161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feb13a-6cad-472b-a59a-0e80733b4d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:06.276009 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-66feb13a-6cad-472b-a59a-0e80733b4d0d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:06.277237 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-66feb13a-6cad-472b-a59a-0e80733b4d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:06.317228 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:06.348920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1004/1991] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:06.358013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4f0b-606b-4db5-8df2-7cdfa4863fa4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:06.358444 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4f0b-606b-4db5-8df2-7cdfa4863fa4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:06.359009 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4f0b-606b-4db5-8df2-7cdfa4863fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:06.383938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-49aa4f0b-606b-4db5-8df2-7cdfa4863fa4 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:06.384282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-49aa4f0b-606b-4db5-8df2-7cdfa4863fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:06.397127 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 988/1992] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:06.406599 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66feb13a-6cad-472b-a59a-0e80733b4d0d None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:06.423695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66feb13a-6cad-472b-a59a-0e80733b4d0d None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:06.428656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d318741-0e0d-4217-a249-827127bc16d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:06.429302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d318741-0e0d-4217-a249-827127bc16d3 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:06.429582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d318741-0e0d-4217-a249-827127bc16d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:06.429892 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d318741-0e0d-4217-a249-827127bc16d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:06.430579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d318741-0e0d-4217-a249-827127bc16d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:52:06.430886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d318741-0e0d-4217-a249-827127bc16d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:06.432686 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d318741-0e0d-4217-a249-827127bc16d3 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:06.447701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1005/1993] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:52:06.454183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4f0b-606b-4db5-8df2-7cdfa4863fa4 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:06.464610 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4f0b-606b-4db5-8df2-7cdfa4863fa4 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:06.469758 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210aa5fa-ff38-401b-b69a-74dca6533860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:06.470410 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210aa5fa-ff38-401b-b69a-74dca6533860 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:06.470665 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210aa5fa-ff38-401b-b69a-74dca6533860 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:06.470936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210aa5fa-ff38-401b-b69a-74dca6533860 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:06.471594 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210aa5fa-ff38-401b-b69a-74dca6533860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:06.473645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210aa5fa-ff38-401b-b69a-74dca6533860 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:06.508028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 989/1994] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:52:06 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:52:06.512873 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d318741-0e0d-4217-a249-827127bc16d3 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:06.529513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d318741-0e0d-4217-a249-827127bc16d3 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:06.610609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b24281-f829-485f-87d6-a512d7f683ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:06.611384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b24281-f829-485f-87d6-a512d7f683ec admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:06.611768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b24281-f829-485f-87d6-a512d7f683ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:06.612287 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b24281-f829-485f-87d6-a512d7f683ec admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:06.613137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b24281-f829-485f-87d6-a512d7f683ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:06.615735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b24281-f829-485f-87d6-a512d7f683ec admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:06.648501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1006/1995] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:52:06 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:52:06.654626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210aa5fa-ff38-401b-b69a-74dca6533860 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:06.665941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210aa5fa-ff38-401b-b69a-74dca6533860 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:06.670989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04258f43-9a6d-4ef2-903d-84f3bda118e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:06.671895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04258f43-9a6d-4ef2-903d-84f3bda118e4 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:06.672236 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04258f43-9a6d-4ef2-903d-84f3bda118e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:06.672567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04258f43-9a6d-4ef2-903d-84f3bda118e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:06.673285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04258f43-9a6d-4ef2-903d-84f3bda118e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:06.675119 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04258f43-9a6d-4ef2-903d-84f3bda118e4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:06.687033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 990/1996] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:52:06.691821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b24281-f829-485f-87d6-a512d7f683ec admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:06.711736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b24281-f829-485f-87d6-a512d7f683ec admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:06.717050 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8cdac-dbc3-4f21-9ab8-b9715b94239b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:06.717662 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8cdac-dbc3-4f21-9ab8-b9715b94239b admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:06.718067 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8cdac-dbc3-4f21-9ab8-b9715b94239b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:06.718558 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8cdac-dbc3-4f21-9ab8-b9715b94239b admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:06.719496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8cdac-dbc3-4f21-9ab8-b9715b94239b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:06.721183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8cdac-dbc3-4f21-9ab8-b9715b94239b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:06.732787 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1007/1997] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:52:06.740670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04258f43-9a6d-4ef2-903d-84f3bda118e4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:06.752607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04258f43-9a6d-4ef2-903d-84f3bda118e4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:06.758056 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f2ee8-738d-4983-86ba-45426006c847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:06.758753 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f2ee8-738d-4983-86ba-45426006c847 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:06.762468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f2ee8-738d-4983-86ba-45426006c847 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:06.762468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f2ee8-738d-4983-86ba-45426006c847 admin admin] PATH_INFO: `/v3/projects/e752c3aef55649ea98ec1a2f3607c27f/users/dc83c1c8fce04c57a685de883344b7e6/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:06.777093 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f2ee8-738d-4983-86ba-45426006c847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e752c3aef55649ea98ec1a2f3607c27f, user_id=dc83c1c8fce04c57a685de883344b7e6, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:06.779839 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f2ee8-738d-4983-86ba-45426006c847 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:06.795796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 991/1998] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:52:06 2022] PUT /identity/v3/projects/e752c3aef55649ea98ec1a2f3607c27f/users/dc83c1c8fce04c57a685de883344b7e6/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:52:06.802926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8448-8aff-4d86-bda6-6d1d9dba4f01 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:06.803282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8448-8aff-4d86-bda6-6d1d9dba4f01 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:06.803625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8448-8aff-4d86-bda6-6d1d9dba4f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:06.825905 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-c8ed8448-8aff-4d86-bda6-6d1d9dba4f01 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:06.827230 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c8ed8448-8aff-4d86-bda6-6d1d9dba4f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:06.897192 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1008/1999] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:52:06.907982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f2ee8-738d-4983-86ba-45426006c847 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:06.909762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-077f2ee8-738d-4983-86ba-45426006c847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:06.985153 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f2ee8-738d-4983-86ba-45426006c847 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:06.990229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7286c0f-ae5f-4aa7-a99e-b355af8e6581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:06.990905 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7286c0f-ae5f-4aa7-a99e-b355af8e6581 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:06.991240 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7286c0f-ae5f-4aa7-a99e-b355af8e6581 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:06.991593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7286c0f-ae5f-4aa7-a99e-b355af8e6581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:07.002556 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7286c0f-ae5f-4aa7-a99e-b355af8e6581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:07.004053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7286c0f-ae5f-4aa7-a99e-b355af8e6581 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:07.015347 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 992/2000] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:52:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:52:13.553253 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d8595-5bcc-4e08-9d44-f822d597403c None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:13.554088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d8595-5bcc-4e08-9d44-f822d597403c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:13.554230 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d8595-5bcc-4e08-9d44-f822d597403c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:13.612771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-b73d8595-5bcc-4e08-9d44-f822d597403c None None] MFA Rules not processed for user `dc83c1c8fce04c57a685de883344b7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:13.614947 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b73d8595-5bcc-4e08-9d44-f822d597403c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:13.667142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:13.759602 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1009/2001] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:13.775184 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7286c0f-ae5f-4aa7-a99e-b355af8e6581 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:13.801240 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7286c0f-ae5f-4aa7-a99e-b355af8e6581 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:13.808760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a469e92-4a23-449a-8b8e-ced1e6af349b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:13.809832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a469e92-4a23-449a-8b8e-ced1e6af349b admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:13.810352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a469e92-4a23-449a-8b8e-ced1e6af349b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:13.810833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a469e92-4a23-449a-8b8e-ced1e6af349b admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:13.812261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a469e92-4a23-449a-8b8e-ced1e6af349b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:13.815822 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a469e92-4a23-449a-8b8e-ced1e6af349b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:13.860982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 993/2002] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:52:13 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:52:13.865498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d8595-5bcc-4e08-9d44-f822d597403c None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:13.894665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d8595-5bcc-4e08-9d44-f822d597403c None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:13.992206 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975bb7c-2677-4239-8eca-265b74daba8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:13.993253 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975bb7c-2677-4239-8eca-265b74daba8c admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:13.993633 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975bb7c-2677-4239-8eca-265b74daba8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:13.993966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975bb7c-2677-4239-8eca-265b74daba8c admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:13.994579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975bb7c-2677-4239-8eca-265b74daba8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:13.998046 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975bb7c-2677-4239-8eca-265b74daba8c admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:14.034273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1010/2003] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:52:13 2022] POST /identity/v3/users => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:52:14.035074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a469e92-4a23-449a-8b8e-ced1e6af349b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:14.055902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a469e92-4a23-449a-8b8e-ced1e6af349b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:14.066065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bee845-e226-4706-8dae-fa54e314dafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:14.067154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bee845-e226-4706-8dae-fa54e314dafa admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:14.067654 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bee845-e226-4706-8dae-fa54e314dafa admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:14.068144 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bee845-e226-4706-8dae-fa54e314dafa admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:14.069271 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bee845-e226-4706-8dae-fa54e314dafa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:14.072474 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bee845-e226-4706-8dae-fa54e314dafa admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:14.095061 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 994/2004] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:52:14.099406 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975bb7c-2677-4239-8eca-265b74daba8c admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:14.113963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975bb7c-2677-4239-8eca-265b74daba8c admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:14.119125 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f99ca-c7ea-4113-b02e-662e57bc9e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:14.119833 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f99ca-c7ea-4113-b02e-662e57bc9e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:14.120154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f99ca-c7ea-4113-b02e-662e57bc9e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:14.121241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f99ca-c7ea-4113-b02e-662e57bc9e89 admin admin] PATH_INFO: `/v3/projects/c15ea1a4d30742a2983b585cf7aa4eff/users/4413b1d895c9438c8f8fe01408e53e8b/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:14.144343 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f99ca-c7ea-4113-b02e-662e57bc9e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15ea1a4d30742a2983b585cf7aa4eff, user_id=4413b1d895c9438c8f8fe01408e53e8b, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:14.149656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f99ca-c7ea-4113-b02e-662e57bc9e89 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:14.179731 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1011/2005] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:52:14 2022] PUT /identity/v3/projects/c15ea1a4d30742a2983b585cf7aa4eff/users/4413b1d895c9438c8f8fe01408e53e8b/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:52:14.185681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bee845-e226-4706-8dae-fa54e314dafa admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:14.195680 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:14.199977 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bee845-e226-4706-8dae-fa54e314dafa admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:14.331082 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da69ad-d449-452f-b98c-688ff64f76f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:14.332721 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da69ad-d449-452f-b98c-688ff64f76f4 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:14.333191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da69ad-d449-452f-b98c-688ff64f76f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:14.333622 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da69ad-d449-452f-b98c-688ff64f76f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:14.334470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da69ad-d449-452f-b98c-688ff64f76f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:14.337927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da69ad-d449-452f-b98c-688ff64f76f4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:14.363724 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 995/2006] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:52:14.365365 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f99ca-c7ea-4113-b02e-662e57bc9e89 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:14.378817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f99ca-c7ea-4113-b02e-662e57bc9e89 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:14.393022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c52c6-8f73-4ccc-8bfb-20367fb33ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:14.393022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c52c6-8f73-4ccc-8bfb-20367fb33ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:14.393022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c52c6-8f73-4ccc-8bfb-20367fb33ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:14.393022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c52c6-8f73-4ccc-8bfb-20367fb33ef7 admin admin] PATH_INFO: `/v3/projects/c15ea1a4d30742a2983b585cf7aa4eff/users/4413b1d895c9438c8f8fe01408e53e8b/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:14.400766 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c52c6-8f73-4ccc-8bfb-20367fb33ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15ea1a4d30742a2983b585cf7aa4eff, user_id=4413b1d895c9438c8f8fe01408e53e8b, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:14.403588 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c52c6-8f73-4ccc-8bfb-20367fb33ef7 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:14.455544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1012/2007] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:52:14 2022] PUT /identity/v3/projects/c15ea1a4d30742a2983b585cf7aa4eff/users/4413b1d895c9438c8f8fe01408e53e8b/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:20.177747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8dee7-0c12-4a46-a37c-3b6a00ad8376 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:20.178968 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8dee7-0c12-4a46-a37c-3b6a00ad8376 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:20.180008 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8dee7-0c12-4a46-a37c-3b6a00ad8376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:20.212184 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a1b8dee7-0c12-4a46-a37c-3b6a00ad8376 None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:20.214070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a1b8dee7-0c12-4a46-a37c-3b6a00ad8376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:20.345929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 996/2008] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:52:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:20.365498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c52c6-8f73-4ccc-8bfb-20367fb33ef7 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:20.373072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-776c52c6-8f73-4ccc-8bfb-20367fb33ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:20.521875 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c52c6-8f73-4ccc-8bfb-20367fb33ef7 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:20.528513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5b3c6-9592-4016-946b-c050880b10cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:20.535189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5b3c6-9592-4016-946b-c050880b10cb service swift] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:20.535748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5b3c6-9592-4016-946b-c050880b10cb service swift] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:20.536650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5b3c6-9592-4016-946b-c050880b10cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:20.557094 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5b3c6-9592-4016-946b-c050880b10cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:20.559644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5b3c6-9592-4016-946b-c050880b10cb service swift] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:20.591657 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1013/2009] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:52:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:52:21.555496 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88398ae2-95c5-4ad6-a40b-9b212aecd3a4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:21.556237 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88398ae2-95c5-4ad6-a40b-9b212aecd3a4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:21.556718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88398ae2-95c5-4ad6-a40b-9b212aecd3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:21.609333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-88398ae2-95c5-4ad6-a40b-9b212aecd3a4 None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:21.611021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-88398ae2-95c5-4ad6-a40b-9b212aecd3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:21.627697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 997/2010] 10.222.0.36 () {58 vars in 967 bytes} [Sat Oct 15 13:52:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:21.652067 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5b3c6-9592-4016-946b-c050880b10cb service swift] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:21.683687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5f2f0-6452-405f-a80b-e04a59529828 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:21.684040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5f2f0-6452-405f-a80b-e04a59529828 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:21.684412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5f2f0-6452-405f-a80b-e04a59529828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:21.690191 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5b3c6-9592-4016-946b-c050880b10cb service swift] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:21.698839 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1787086-4a74-4087-8d12-e49e4f19701a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:21.700782 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1787086-4a74-4087-8d12-e49e4f19701a service swift] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:21.701062 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1787086-4a74-4087-8d12-e49e4f19701a service swift] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:21.701437 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1787086-4a74-4087-8d12-e49e4f19701a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:21.716673 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1787086-4a74-4087-8d12-e49e4f19701a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:21.718725 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1787086-4a74-4087-8d12-e49e4f19701a service swift] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:21.733204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1014/2011] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:52:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:52:21.757110 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e3a5f2f0-6452-405f-a80b-e04a59529828 None None] MFA Rules not processed for user `4413b1d895c9438c8f8fe01408e53e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:21.758572 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e3a5f2f0-6452-405f-a80b-e04a59529828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:21.982935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 998/2012] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:21.998359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1787086-4a74-4087-8d12-e49e4f19701a service swift] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:22.002468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f1787086-4a74-4087-8d12-e49e4f19701a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:22.016568 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:22.217389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1787086-4a74-4087-8d12-e49e4f19701a service swift] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:22.226526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85417dd6-a4f2-436c-83c3-945dd5ec1b64 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f5aabcb1414413a8dce7b76ce701582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:22.227269 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85417dd6-a4f2-436c-83c3-945dd5ec1b64 service glance] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:22.227550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85417dd6-a4f2-436c-83c3-945dd5ec1b64 service glance] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:22.227959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85417dd6-a4f2-436c-83c3-945dd5ec1b64 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:22.372493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b81398-c08b-4764-836e-511eb9e220a4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:22.372770 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b81398-c08b-4764-836e-511eb9e220a4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:22.373310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b81398-c08b-4764-836e-511eb9e220a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:22.389354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85417dd6-a4f2-436c-83c3-945dd5ec1b64 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:22.394059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85417dd6-a4f2-436c-83c3-945dd5ec1b64 service glance] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:22.426960 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-06b81398-c08b-4764-836e-511eb9e220a4 None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:22.429326 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-06b81398-c08b-4764-836e-511eb9e220a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:22.437208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1015/2013] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:52:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:52:22.447213 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 999/2014] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:52:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 15 13:52:22.460585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85417dd6-a4f2-436c-83c3-945dd5ec1b64 service glance] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:22.494364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85417dd6-a4f2-436c-83c3-945dd5ec1b64 service glance] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:22.509067 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdebef-35b4-4781-b799-5cc8f81baccd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:22.510410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdebef-35b4-4781-b799-5cc8f81baccd service swift] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:22.510899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdebef-35b4-4781-b799-5cc8f81baccd service swift] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:22.511349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdebef-35b4-4781-b799-5cc8f81baccd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:22.530919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdebef-35b4-4781-b799-5cc8f81baccd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:22.533001 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdebef-35b4-4781-b799-5cc8f81baccd service swift] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:22.554402 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1016/2015] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:52:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 13:52:22.709281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279d5b-e97c-4f85-80d2-663181b3a045 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:22.710168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279d5b-e97c-4f85-80d2-663181b3a045 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:22.710744 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279d5b-e97c-4f85-80d2-663181b3a045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:22.750195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-46279d5b-e97c-4f85-80d2-663181b3a045 None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:22.751577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-46279d5b-e97c-4f85-80d2-663181b3a045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:22.771951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1000/2016] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:52:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:22.790815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdebef-35b4-4781-b799-5cc8f81baccd service swift] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:22.815482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdebef-35b4-4781-b799-5cc8f81baccd service swift] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:22.827841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a55e06-7345-419c-a407-060d6f0ea270 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:22.828915 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a55e06-7345-419c-a407-060d6f0ea270 service swift] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:22.829509 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a55e06-7345-419c-a407-060d6f0ea270 service swift] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:22.830064 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a55e06-7345-419c-a407-060d6f0ea270 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:22.846566 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a55e06-7345-419c-a407-060d6f0ea270 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:22.848959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a55e06-7345-419c-a407-060d6f0ea270 service swift] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:22.872368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1017/2017] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:52:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:52:23.581987 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7195297-db6b-42e5-9df9-1d1b580edd93 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:23.582392 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7195297-db6b-42e5-9df9-1d1b580edd93 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:23.582866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7195297-db6b-42e5-9df9-1d1b580edd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:23.625660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a7195297-db6b-42e5-9df9-1d1b580edd93 None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:23.627806 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a7195297-db6b-42e5-9df9-1d1b580edd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:23.641027 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1001/2018] 10.222.0.36 () {58 vars in 967 bytes} [Sat Oct 15 13:52:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:23.675675 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a55e06-7345-419c-a407-060d6f0ea270 service swift] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:23.711932 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a55e06-7345-419c-a407-060d6f0ea270 service swift] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:23.740363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399bb7dd-d6ef-4960-ab33-c3035a8e1ca1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:23.743124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bb7dd-d6ef-4960-ab33-c3035a8e1ca1 service swift] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:23.744872 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bb7dd-d6ef-4960-ab33-c3035a8e1ca1 service swift] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:23.746155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bb7dd-d6ef-4960-ab33-c3035a8e1ca1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:23.792013 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399bb7dd-d6ef-4960-ab33-c3035a8e1ca1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:23.800259 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399bb7dd-d6ef-4960-ab33-c3035a8e1ca1 service swift] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:23.825076 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1018/2019] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:52:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:52:31.448358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7195297-db6b-42e5-9df9-1d1b580edd93 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:31.451378 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a7195297-db6b-42e5-9df9-1d1b580edd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:31.459832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:31.564436 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7195297-db6b-42e5-9df9-1d1b580edd93 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:31.569747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632ceaa-b102-4f6c-a44d-a3ad4f1fe990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:31.570407 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632ceaa-b102-4f6c-a44d-a3ad4f1fe990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:31.570679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632ceaa-b102-4f6c-a44d-a3ad4f1fe990 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:31.570945 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632ceaa-b102-4f6c-a44d-a3ad4f1fe990 admin admin] PATH_INFO: `/v3/users/a02652d1b503426cac6e77aa68f2e98c` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:31.572341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632ceaa-b102-4f6c-a44d-a3ad4f1fe990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a02652d1b503426cac6e77aa68f2e98c)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:31.575222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632ceaa-b102-4f6c-a44d-a3ad4f1fe990 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:31.664788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1632ceaa-b102-4f6c-a44d-a3ad4f1fe990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a02652d1b503426cac6e77aa68f2e98c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:31.665136 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1632ceaa-b102-4f6c-a44d-a3ad4f1fe990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a02652d1b503426cac6e77aa68f2e98c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:31.665848 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1632ceaa-b102-4f6c-a44d-a3ad4f1fe990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a02652d1b503426cac6e77aa68f2e98c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:31.701135 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1632ceaa-b102-4f6c-a44d-a3ad4f1fe990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a02652d1b503426cac6e77aa68f2e98c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:31.721948 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1632ceaa-b102-4f6c-a44d-a3ad4f1fe990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a02652d1b503426cac6e77aa68f2e98c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:31.749117 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1002/2020] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:52:31 2022] DELETE /identity/v3/users/a02652d1b503426cac6e77aa68f2e98c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:31.754158 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd62fcb-4b65-4dd8-a488-6130a8a1acfb None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:31.755095 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd62fcb-4b65-4dd8-a488-6130a8a1acfb None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:31.755519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd62fcb-4b65-4dd8-a488-6130a8a1acfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:31.777945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-6fd62fcb-4b65-4dd8-a488-6130a8a1acfb None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:31.779177 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6fd62fcb-4b65-4dd8-a488-6130a8a1acfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:31.852650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1019/2021] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:31.863070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632ceaa-b102-4f6c-a44d-a3ad4f1fe990 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:31.864581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1632ceaa-b102-4f6c-a44d-a3ad4f1fe990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:31.955649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632ceaa-b102-4f6c-a44d-a3ad4f1fe990 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:31.960997 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b65f96-ccd1-4e55-9cef-b7cbdba9a93a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:31.961822 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b65f96-ccd1-4e55-9cef-b7cbdba9a93a service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:31.962089 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b65f96-ccd1-4e55-9cef-b7cbdba9a93a service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:31.962358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b65f96-ccd1-4e55-9cef-b7cbdba9a93a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:31.974746 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b65f96-ccd1-4e55-9cef-b7cbdba9a93a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:31.976274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b65f96-ccd1-4e55-9cef-b7cbdba9a93a service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:32.001955 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1003/2022] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:52:32.270576 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd62fcb-4b65-4dd8-a488-6130a8a1acfb None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:32.272659 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6fd62fcb-4b65-4dd8-a488-6130a8a1acfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:32.333323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd62fcb-4b65-4dd8-a488-6130a8a1acfb None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:32.351585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:32.355031 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:32.356214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:32.357414 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] PATH_INFO: `/v3/projects/9da8e5e0fae247f48af6d303ae2fcdff` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:32.365632 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9da8e5e0fae247f48af6d303ae2fcdff)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:32.371168 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:32.390435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:32.473316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9da8e5e0fae247f48af6d303ae2fcdff', 'request_id': 'req-80afce13-8235-4b4a-832d-d6e1c7de0e34'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:32.494200 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9da8e5e0fae247f48af6d303ae2fcdff', 'request_id': 'req-80afce13-8235-4b4a-832d-d6e1c7de0e34'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:32.510271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9da8e5e0fae247f48af6d303ae2fcdff', 'request_id': 'req-80afce13-8235-4b4a-832d-d6e1c7de0e34'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:32.537085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] The token cache is being invalidate because project 9da8e5e0fae247f48af6d303ae2fcdff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:52:32.537550 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:32.540261 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:32.557808 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1020/2023] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:52:32 2022] DELETE /identity/v3/projects/9da8e5e0fae247f48af6d303ae2fcdff => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:34.635558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee485f-8f5e-42ab-b469-525ec6d190bc None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:34.636775 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee485f-8f5e-42ab-b469-525ec6d190bc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:34.637356 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee485f-8f5e-42ab-b469-525ec6d190bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:34.658971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-32ee485f-8f5e-42ab-b469-525ec6d190bc None None] MFA Rules not processed for user `dc83c1c8fce04c57a685de883344b7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:34.660473 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-32ee485f-8f5e-42ab-b469-525ec6d190bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:34.682635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:34.737433 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1004/2024] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:36.130544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:36.132270 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:36.236927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afce13-8235-4b4a-832d-d6e1c7de0e34 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:36.249599 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecfd92-23fd-452b-aa0b-bc4fda2e7792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:36.250533 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecfd92-23fd-452b-aa0b-bc4fda2e7792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:36.250750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecfd92-23fd-452b-aa0b-bc4fda2e7792 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:36.251071 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecfd92-23fd-452b-aa0b-bc4fda2e7792 admin admin] PATH_INFO: `/v3/users/0abbd790650f40a6b89b3179c2078f88` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:36.252709 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecfd92-23fd-452b-aa0b-bc4fda2e7792 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0abbd790650f40a6b89b3179c2078f88)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:36.261771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecfd92-23fd-452b-aa0b-bc4fda2e7792 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:36.329511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-76ecfd92-23fd-452b-aa0b-bc4fda2e7792 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0abbd790650f40a6b89b3179c2078f88'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:36.337351 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-76ecfd92-23fd-452b-aa0b-bc4fda2e7792 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0abbd790650f40a6b89b3179c2078f88'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:36.338284 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-76ecfd92-23fd-452b-aa0b-bc4fda2e7792 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0abbd790650f40a6b89b3179c2078f88'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:36.344675 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-76ecfd92-23fd-452b-aa0b-bc4fda2e7792 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0abbd790650f40a6b89b3179c2078f88'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:36.358292 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-76ecfd92-23fd-452b-aa0b-bc4fda2e7792 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0abbd790650f40a6b89b3179c2078f88'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:36.381680 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1021/2025] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:52:36 2022] DELETE /identity/v3/users/0abbd790650f40a6b89b3179c2078f88 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:36.386477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77c243-82b8-4b9e-87bd-821f4d76be73 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:36.387036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77c243-82b8-4b9e-87bd-821f4d76be73 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:36.387036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77c243-82b8-4b9e-87bd-821f4d76be73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:36.415602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-cf77c243-82b8-4b9e-87bd-821f4d76be73 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:36.416965 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cf77c243-82b8-4b9e-87bd-821f4d76be73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:36.483443 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1005/2026] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:36.495966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecfd92-23fd-452b-aa0b-bc4fda2e7792 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:36.498391 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-76ecfd92-23fd-452b-aa0b-bc4fda2e7792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:36.546114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:36.586480 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecfd92-23fd-452b-aa0b-bc4fda2e7792 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:36.591613 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34276892-65eb-46a1-87ff-5b073984c2f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:36.592356 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34276892-65eb-46a1-87ff-5b073984c2f0 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:36.592638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34276892-65eb-46a1-87ff-5b073984c2f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:36.593005 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34276892-65eb-46a1-87ff-5b073984c2f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:36.608229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34276892-65eb-46a1-87ff-5b073984c2f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:36.610170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34276892-65eb-46a1-87ff-5b073984c2f0 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:36.624854 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1022/2027] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:52:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:52:37.102124 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77c243-82b8-4b9e-87bd-821f4d76be73 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:37.103968 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cf77c243-82b8-4b9e-87bd-821f4d76be73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:37.122699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77c243-82b8-4b9e-87bd-821f4d76be73 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:37.129870 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0855f-21e9-4e99-a17a-b2638001a319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:37.131254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34276892-65eb-46a1-87ff-5b073984c2f0 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:37.131773 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0855f-21e9-4e99-a17a-b2638001a319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:37.132579 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0855f-21e9-4e99-a17a-b2638001a319 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:37.133243 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0855f-21e9-4e99-a17a-b2638001a319 admin admin] PATH_INFO: `/v3/users/16e40bc5611048a59f3ba09b715985cb` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:37.133531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-34276892-65eb-46a1-87ff-5b073984c2f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:37.134998 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0855f-21e9-4e99-a17a-b2638001a319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16e40bc5611048a59f3ba09b715985cb)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:37.137512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0855f-21e9-4e99-a17a-b2638001a319 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:37.158245 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34276892-65eb-46a1-87ff-5b073984c2f0 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:37.177499 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a032a-5524-4c61-9bbc-7098741af6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:37.178313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a032a-5524-4c61-9bbc-7098741af6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:37.178719 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a032a-5524-4c61-9bbc-7098741af6da admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:37.179089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a032a-5524-4c61-9bbc-7098741af6da admin admin] PATH_INFO: `/v3/projects/60710b498def444ab00751cf12ef8940` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:37.180569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a032a-5524-4c61-9bbc-7098741af6da admin admin] RBAC: Authorizing `identity:delete_project(project_id=60710b498def444ab00751cf12ef8940)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:37.184166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a032a-5524-4c61-9bbc-7098741af6da admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:37.274742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2bc0855f-21e9-4e99-a17a-b2638001a319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16e40bc5611048a59f3ba09b715985cb'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:37.277000 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2bc0855f-21e9-4e99-a17a-b2638001a319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16e40bc5611048a59f3ba09b715985cb'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:37.278590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2bc0855f-21e9-4e99-a17a-b2638001a319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16e40bc5611048a59f3ba09b715985cb'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:37.285213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5b6a032a-5524-4c61-9bbc-7098741af6da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60710b498def444ab00751cf12ef8940', 'request_id': 'req-5b6a032a-5524-4c61-9bbc-7098741af6da'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:37.297336 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5b6a032a-5524-4c61-9bbc-7098741af6da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60710b498def444ab00751cf12ef8940', 'request_id': 'req-5b6a032a-5524-4c61-9bbc-7098741af6da'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:37.313555 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5b6a032a-5524-4c61-9bbc-7098741af6da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60710b498def444ab00751cf12ef8940', 'request_id': 'req-5b6a032a-5524-4c61-9bbc-7098741af6da'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:37.318543 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2bc0855f-21e9-4e99-a17a-b2638001a319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16e40bc5611048a59f3ba09b715985cb'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:37.332077 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5b6a032a-5524-4c61-9bbc-7098741af6da admin admin] The token cache is being invalidate because project 60710b498def444ab00751cf12ef8940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:52:37.332815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5b6a032a-5524-4c61-9bbc-7098741af6da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:37.334201 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5b6a032a-5524-4c61-9bbc-7098741af6da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:37.336589 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1023/2028] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:52:37 2022] DELETE /identity/v3/projects/60710b498def444ab00751cf12ef8940 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:37.336799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2bc0855f-21e9-4e99-a17a-b2638001a319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16e40bc5611048a59f3ba09b715985cb'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:37.344211 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b43d60-3f39-45bf-b761-00dfbe8892e8 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:37.344818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b43d60-3f39-45bf-b761-00dfbe8892e8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:37.345293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b43d60-3f39-45bf-b761-00dfbe8892e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:37.373658 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1006/2029] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:52:37 2022] DELETE /identity/v3/users/16e40bc5611048a59f3ba09b715985cb => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:37.376509 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-76b43d60-3f39-45bf-b761-00dfbe8892e8 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:37.381000 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-76b43d60-3f39-45bf-b761-00dfbe8892e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:37.383139 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47e209-a211-4623-92e2-ab6459b3371b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:37.383644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47e209-a211-4623-92e2-ab6459b3371b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:37.384108 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47e209-a211-4623-92e2-ab6459b3371b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:37.419890 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6e47e209-a211-4623-92e2-ab6459b3371b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:37.425682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6e47e209-a211-4623-92e2-ab6459b3371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:37.518555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:37.525058 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1024/2030] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:37.534290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fc036-1da8-4162-a457-5d08521ad98d None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:37.535053 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fc036-1da8-4162-a457-5d08521ad98d None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:37.535489 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fc036-1da8-4162-a457-5d08521ad98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:37.556269 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1007/2031] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:52:37.564625 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47e209-a211-4623-92e2-ab6459b3371b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:37.566523 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6e47e209-a211-4623-92e2-ab6459b3371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:37.570048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-e89fc036-1da8-4162-a457-5d08521ad98d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:37.573427 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e89fc036-1da8-4162-a457-5d08521ad98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:37.602144 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1025/2032] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:37.608346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fc036-1da8-4162-a457-5d08521ad98d None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:37.623954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fc036-1da8-4162-a457-5d08521ad98d None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:37.631964 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac0fb0-48ba-407c-96d1-938e1a340091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:37.633017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac0fb0-48ba-407c-96d1-938e1a340091 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:37.633500 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac0fb0-48ba-407c-96d1-938e1a340091 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:37.633977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac0fb0-48ba-407c-96d1-938e1a340091 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:37.635490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac0fb0-48ba-407c-96d1-938e1a340091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:52:37.636058 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac0fb0-48ba-407c-96d1-938e1a340091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:37.639254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac0fb0-48ba-407c-96d1-938e1a340091 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:37.663761 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1026/2033] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:52:37.669205 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac0fb0-48ba-407c-96d1-938e1a340091 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:37.688575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47e209-a211-4623-92e2-ab6459b3371b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:37.689030 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac0fb0-48ba-407c-96d1-938e1a340091 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:37.692569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084efcab-0ded-4832-ba93-a12f01e09607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:37.693321 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084efcab-0ded-4832-ba93-a12f01e09607 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:37.693663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084efcab-0ded-4832-ba93-a12f01e09607 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:37.694001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084efcab-0ded-4832-ba93-a12f01e09607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:37.700665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e264bb-ceb8-45a5-8585-4a8f3763d486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:37.701315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e264bb-ceb8-45a5-8585-4a8f3763d486 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:37.701597 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e264bb-ceb8-45a5-8585-4a8f3763d486 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:37.701874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e264bb-ceb8-45a5-8585-4a8f3763d486 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:37.702579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e264bb-ceb8-45a5-8585-4a8f3763d486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:37.704879 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e264bb-ceb8-45a5-8585-4a8f3763d486 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:37.706204 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084efcab-0ded-4832-ba93-a12f01e09607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:37.708165 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084efcab-0ded-4832-ba93-a12f01e09607 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:37.721282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1008/2034] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:52:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:52:37.747955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1027/2035] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:52:37 2022] POST /identity/v3/projects => generated 399 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:52:37.750567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084efcab-0ded-4832-ba93-a12f01e09607 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:37.779929 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084efcab-0ded-4832-ba93-a12f01e09607 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:37.857413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661071d-aa79-4d19-ab45-d2cb1271be3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:37.858045 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661071d-aa79-4d19-ab45-d2cb1271be3d admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:37.858398 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661071d-aa79-4d19-ab45-d2cb1271be3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:37.858618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661071d-aa79-4d19-ab45-d2cb1271be3d admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:37.860012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661071d-aa79-4d19-ab45-d2cb1271be3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:37.862134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661071d-aa79-4d19-ab45-d2cb1271be3d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:37.899626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1009/2036] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:52:37 2022] POST /identity/v3/users => generated 300 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:52:37.904078 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e264bb-ceb8-45a5-8585-4a8f3763d486 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:37.915906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e264bb-ceb8-45a5-8585-4a8f3763d486 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:37.921160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45237c-3fbc-4fae-9b23-b9ab319cd942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:37.922438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45237c-3fbc-4fae-9b23-b9ab319cd942 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:37.922721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45237c-3fbc-4fae-9b23-b9ab319cd942 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:37.922999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45237c-3fbc-4fae-9b23-b9ab319cd942 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:37.923717 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45237c-3fbc-4fae-9b23-b9ab319cd942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:37.925380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45237c-3fbc-4fae-9b23-b9ab319cd942 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:37.926930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:37.947814 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1028/2037] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:52:37.948851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661071d-aa79-4d19-ab45-d2cb1271be3d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:37.960261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661071d-aa79-4d19-ab45-d2cb1271be3d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:37.964681 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e686c8-04a9-49bb-aa29-2cc3b2265dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:37.965316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e686c8-04a9-49bb-aa29-2cc3b2265dab admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:37.965592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e686c8-04a9-49bb-aa29-2cc3b2265dab admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:37.965861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e686c8-04a9-49bb-aa29-2cc3b2265dab admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:37.966506 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e686c8-04a9-49bb-aa29-2cc3b2265dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:37.968254 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e686c8-04a9-49bb-aa29-2cc3b2265dab admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:37.983930 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1010/2038] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:52:37.987227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45237c-3fbc-4fae-9b23-b9ab319cd942 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:38.005229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45237c-3fbc-4fae-9b23-b9ab319cd942 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:38.018494 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35ec82-35c5-4263-af76-5782dfc0aa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:38.019551 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35ec82-35c5-4263-af76-5782dfc0aa68 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:38.020063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35ec82-35c5-4263-af76-5782dfc0aa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:38.020524 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35ec82-35c5-4263-af76-5782dfc0aa68 admin admin] PATH_INFO: `/v3/projects/c40b016a3d634d829eb43b9e6bed02a2/users/da7b3e62c94f4e499522eab50119df9e/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:38.038656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35ec82-35c5-4263-af76-5782dfc0aa68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c40b016a3d634d829eb43b9e6bed02a2, user_id=da7b3e62c94f4e499522eab50119df9e, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:38.041632 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35ec82-35c5-4263-af76-5782dfc0aa68 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:38.066747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e686c8-04a9-49bb-aa29-2cc3b2265dab admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:38.069414 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-89e686c8-04a9-49bb-aa29-2cc3b2265dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:38.071620 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1029/2039] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:52:37 2022] PUT /identity/v3/projects/c40b016a3d634d829eb43b9e6bed02a2/users/da7b3e62c94f4e499522eab50119df9e/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:38.084710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a1ea3-8a31-4dc6-9c3c-eac890a2dabb None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:38.085245 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a1ea3-8a31-4dc6-9c3c-eac890a2dabb None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:38.085744 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a1ea3-8a31-4dc6-9c3c-eac890a2dabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:38.091664 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e686c8-04a9-49bb-aa29-2cc3b2265dab admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:38.127218 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-312a1ea3-8a31-4dc6-9c3c-eac890a2dabb None None] MFA Rules not processed for user `da7b3e62c94f4e499522eab50119df9e`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:38.128716 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-312a1ea3-8a31-4dc6-9c3c-eac890a2dabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:38.205923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d630a-6ada-4854-85ab-52f5c3a2018d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:38.207352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d630a-6ada-4854-85ab-52f5c3a2018d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:38.207843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d630a-6ada-4854-85ab-52f5c3a2018d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:38.208325 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d630a-6ada-4854-85ab-52f5c3a2018d admin admin] PATH_INFO: `/v3/projects/4172b607df024042948dc4af055677f8` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:38.209850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d630a-6ada-4854-85ab-52f5c3a2018d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4172b607df024042948dc4af055677f8)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:38.212368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d630a-6ada-4854-85ab-52f5c3a2018d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:38.222761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:38.254731 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1030/2040] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:38 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:38.274094 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a1ea3-8a31-4dc6-9c3c-eac890a2dabb None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:38.276621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-312a1ea3-8a31-4dc6-9c3c-eac890a2dabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:38.296779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-754d630a-6ada-4854-85ab-52f5c3a2018d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4172b607df024042948dc4af055677f8', 'request_id': 'req-754d630a-6ada-4854-85ab-52f5c3a2018d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:38.313390 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-754d630a-6ada-4854-85ab-52f5c3a2018d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4172b607df024042948dc4af055677f8', 'request_id': 'req-754d630a-6ada-4854-85ab-52f5c3a2018d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:38.329740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-754d630a-6ada-4854-85ab-52f5c3a2018d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4172b607df024042948dc4af055677f8', 'request_id': 'req-754d630a-6ada-4854-85ab-52f5c3a2018d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:38.348329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-754d630a-6ada-4854-85ab-52f5c3a2018d admin admin] The token cache is being invalidate because project 4172b607df024042948dc4af055677f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:52:38.348927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-754d630a-6ada-4854-85ab-52f5c3a2018d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:38.349992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-754d630a-6ada-4854-85ab-52f5c3a2018d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:38.351938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1011/2041] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:52:38 2022] DELETE /identity/v3/projects/4172b607df024042948dc4af055677f8 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:38.357279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1074d8-32d0-46d2-a4ce-f53ac8c8b13f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:38.357559 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1074d8-32d0-46d2-a4ce-f53ac8c8b13f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:38.357829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1074d8-32d0-46d2-a4ce-f53ac8c8b13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:38.364872 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a1ea3-8a31-4dc6-9c3c-eac890a2dabb None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:38.412379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-3d1074d8-32d0-46d2-a4ce-f53ac8c8b13f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:38.413614 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3d1074d8-32d0-46d2-a4ce-f53ac8c8b13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:38.437849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97166fc3-31a3-4c8c-ba00-f0f60235cd06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:38.438713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97166fc3-31a3-4c8c-ba00-f0f60235cd06 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:38.439139 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97166fc3-31a3-4c8c-ba00-f0f60235cd06 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:38.443610 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97166fc3-31a3-4c8c-ba00-f0f60235cd06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:38.446612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-97166fc3-31a3-4c8c-ba00-f0f60235cd06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:38.493470 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:38.526686 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1012/2042] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:52:38.528465 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90c7a-6bbb-4701-b75e-27d8c7c5fd55 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:38.528743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90c7a-6bbb-4701-b75e-27d8c7c5fd55 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:38.529016 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90c7a-6bbb-4701-b75e-27d8c7c5fd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:38.555485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d1a90c7a-6bbb-4701-b75e-27d8c7c5fd55 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:38.557024 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d1a90c7a-6bbb-4701-b75e-27d8c7c5fd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:38.567816 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1013/2043] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:38.572157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a90c7a-6bbb-4701-b75e-27d8c7c5fd55 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:38.581448 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97166fc3-31a3-4c8c-ba00-f0f60235cd06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:38.584640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97166fc3-31a3-4c8c-ba00-f0f60235cd06 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:38.584817 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a90c7a-6bbb-4701-b75e-27d8c7c5fd55 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:38.595288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9f15b-1e38-44fa-a1f4-9c009b54cf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:38.596497 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9f15b-1e38-44fa-a1f4-9c009b54cf29 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:38.596950 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9f15b-1e38-44fa-a1f4-9c009b54cf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:38.597514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9f15b-1e38-44fa-a1f4-9c009b54cf29 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:38.598220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9f15b-1e38-44fa-a1f4-9c009b54cf29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:52:38.598518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9f15b-1e38-44fa-a1f4-9c009b54cf29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:38.601319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9f15b-1e38-44fa-a1f4-9c009b54cf29 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:38.623933 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1014/2044] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 13:52:38.628460 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1031/2045] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:52:38 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:52:38.629215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9f15b-1e38-44fa-a1f4-9c009b54cf29 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:38.643202 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9f15b-1e38-44fa-a1f4-9c009b54cf29 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:38.649513 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0b446-dcb8-4152-bdc6-1071088c6dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:38.649711 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cc1d8-913b-4667-9e81-bcce719e1a8b None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:38.650210 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cc1d8-913b-4667-9e81-bcce719e1a8b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:38.650471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0b446-dcb8-4152-bdc6-1071088c6dde admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:38.650665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cc1d8-913b-4667-9e81-bcce719e1a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:38.650996 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0b446-dcb8-4152-bdc6-1071088c6dde admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:38.653826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0b446-dcb8-4152-bdc6-1071088c6dde admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:38.653826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0b446-dcb8-4152-bdc6-1071088c6dde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:38.655394 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0b446-dcb8-4152-bdc6-1071088c6dde admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:38.689847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-617cc1d8-913b-4667-9e81-bcce719e1a8b None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:38.692255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-617cc1d8-913b-4667-9e81-bcce719e1a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:38.709434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1015/2046] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:52:38 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:52:38.715646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0b446-dcb8-4152-bdc6-1071088c6dde admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:38.824805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1032/2047] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:52:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:52:38.839282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cc1d8-913b-4667-9e81-bcce719e1a8b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:38.841192 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-617cc1d8-913b-4667-9e81-bcce719e1a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:38.865060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:38.893343 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0b446-dcb8-4152-bdc6-1071088c6dde admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:38.907397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a928c04-f654-4543-a73e-d7aaaf11c12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:38.908076 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a928c04-f654-4543-a73e-d7aaaf11c12c admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:38.908436 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a928c04-f654-4543-a73e-d7aaaf11c12c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:38.913240 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a928c04-f654-4543-a73e-d7aaaf11c12c admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:38.913957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a928c04-f654-4543-a73e-d7aaaf11c12c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:38.917378 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a928c04-f654-4543-a73e-d7aaaf11c12c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:38.971548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1016/2048] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:52:38 2022] POST /identity/v3/users => generated 287 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:52:38.976531 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a928c04-f654-4543-a73e-d7aaaf11c12c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:38.993326 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a928c04-f654-4543-a73e-d7aaaf11c12c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:38.999350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cc1d8-913b-4667-9e81-bcce719e1a8b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:39.005620 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2c69d-debe-4e55-b68e-30071c055342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:39.006261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2c69d-debe-4e55-b68e-30071c055342 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:39.006536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2c69d-debe-4e55-b68e-30071c055342 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:39.006800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2c69d-debe-4e55-b68e-30071c055342 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:39.008228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2c69d-debe-4e55-b68e-30071c055342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:39.015677 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2c69d-debe-4e55-b68e-30071c055342 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:39.041721 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1017/2049] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:52:39.043395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2c69d-debe-4e55-b68e-30071c055342 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:39.062837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2c69d-debe-4e55-b68e-30071c055342 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:39.069465 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb1d52-b740-4630-abbe-b7c87c803697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:39.071020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb1d52-b740-4630-abbe-b7c87c803697 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:39.071469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb1d52-b740-4630-abbe-b7c87c803697 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:39.071936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb1d52-b740-4630-abbe-b7c87c803697 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:39.073033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb1d52-b740-4630-abbe-b7c87c803697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:39.075617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb1d52-b740-4630-abbe-b7c87c803697 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:39.092023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1018/2050] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:52:39.093226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb1d52-b740-4630-abbe-b7c87c803697 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:39.113952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb1d52-b740-4630-abbe-b7c87c803697 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:39.125044 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a6197-3aaa-4348-9bfc-e3b6bf122b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:39.127779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a6197-3aaa-4348-9bfc-e3b6bf122b89 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:39.128303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a6197-3aaa-4348-9bfc-e3b6bf122b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:39.128900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a6197-3aaa-4348-9bfc-e3b6bf122b89 admin admin] PATH_INFO: `/v3/projects/e39808ea214944b6ab06774fa1b501e7/users/56b68e26810b4329901a45db6a23d262/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:39.132507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b861b-108c-401b-bde2-147d4b759616 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:39.133142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b861b-108c-401b-bde2-147d4b759616 service swift] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:39.133501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b861b-108c-401b-bde2-147d4b759616 service swift] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:39.134565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b861b-108c-401b-bde2-147d4b759616 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:39.154086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a6197-3aaa-4348-9bfc-e3b6bf122b89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e39808ea214944b6ab06774fa1b501e7, user_id=56b68e26810b4329901a45db6a23d262, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:39.157256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a6197-3aaa-4348-9bfc-e3b6bf122b89 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:39.164696 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:39.182544 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1019/2051] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:52:39 2022] PUT /identity/v3/projects/e39808ea214944b6ab06774fa1b501e7/users/56b68e26810b4329901a45db6a23d262/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:39.194515 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a96648-ac61-44b8-bd68-7e02bbeb4aab None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:39.194962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a96648-ac61-44b8-bd68-7e02bbeb4aab None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:39.195378 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a96648-ac61-44b8-bd68-7e02bbeb4aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:39.236844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b861b-108c-401b-bde2-147d4b759616 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:39.238994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b861b-108c-401b-bde2-147d4b759616 service swift] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:39.239832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b3a96648-ac61-44b8-bd68-7e02bbeb4aab None None] MFA Rules not processed for user `56b68e26810b4329901a45db6a23d262`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:39.241577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b3a96648-ac61-44b8-bd68-7e02bbeb4aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:39.252668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1033/2052] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:52:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 13:52:39.387267 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1020/2053] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:52:39.388271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b861b-108c-401b-bde2-147d4b759616 service swift] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:39.406577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b861b-108c-401b-bde2-147d4b759616 service swift] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:39.586026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa99cf2-bdf2-4844-b4bc-821c486be605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:39.601592 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa99cf2-bdf2-4844-b4bc-821c486be605 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:39.604159 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa99cf2-bdf2-4844-b4bc-821c486be605 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:39.604456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa99cf2-bdf2-4844-b4bc-821c486be605 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:39.609117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa99cf2-bdf2-4844-b4bc-821c486be605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:39.611521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa99cf2-bdf2-4844-b4bc-821c486be605 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:39.692018 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1034/2054] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:52:39 2022] POST /identity/v3/projects => generated 377 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:52:39.697925 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a96648-ac61-44b8-bd68-7e02bbeb4aab None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:39.721985 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a96648-ac61-44b8-bd68-7e02bbeb4aab None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:39.885037 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186647f-2326-44b5-9b09-a6db86d73cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:39.885683 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186647f-2326-44b5-9b09-a6db86d73cde admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:39.885959 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186647f-2326-44b5-9b09-a6db86d73cde admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:39.886235 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186647f-2326-44b5-9b09-a6db86d73cde admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:39.887132 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186647f-2326-44b5-9b09-a6db86d73cde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:39.889199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186647f-2326-44b5-9b09-a6db86d73cde admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:39.954112 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1021/2055] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:52:39 2022] POST /identity/v3/users => generated 289 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:52:39.958266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa99cf2-bdf2-4844-b4bc-821c486be605 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:39.981675 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa99cf2-bdf2-4844-b4bc-821c486be605 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:39.994292 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03f553-059c-4d8c-b41f-1b95ac3b0bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:39.995384 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03f553-059c-4d8c-b41f-1b95ac3b0bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:39.996015 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03f553-059c-4d8c-b41f-1b95ac3b0bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:39.996586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03f553-059c-4d8c-b41f-1b95ac3b0bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:39.997843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03f553-059c-4d8c-b41f-1b95ac3b0bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:40.002077 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03f553-059c-4d8c-b41f-1b95ac3b0bf1 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:40.049080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1035/2056] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:52:40.049659 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186647f-2326-44b5-9b09-a6db86d73cde admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:40.083593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186647f-2326-44b5-9b09-a6db86d73cde admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:40.106868 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f091f-3715-4f75-8388-f694a90795e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:40.108219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f091f-3715-4f75-8388-f694a90795e6 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:40.108524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f091f-3715-4f75-8388-f694a90795e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:40.109688 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f091f-3715-4f75-8388-f694a90795e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:40.110718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f091f-3715-4f75-8388-f694a90795e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:40.114032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f091f-3715-4f75-8388-f694a90795e6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:40.151228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1022/2057] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:52:40.155907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03f553-059c-4d8c-b41f-1b95ac3b0bf1 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:40.173347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03f553-059c-4d8c-b41f-1b95ac3b0bf1 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:40.197780 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfa081-b904-410a-afe2-0a2b704d9297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:40.200744 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfa081-b904-410a-afe2-0a2b704d9297 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:40.201031 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfa081-b904-410a-afe2-0a2b704d9297 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:40.201313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfa081-b904-410a-afe2-0a2b704d9297 admin admin] PATH_INFO: `/v3/projects/957eeec9fcb7469d92262dd5446f6986/users/44bfdeefdba14289a7be2e9356346f34/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:40.251267 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfa081-b904-410a-afe2-0a2b704d9297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=957eeec9fcb7469d92262dd5446f6986, user_id=44bfdeefdba14289a7be2e9356346f34, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:40.260184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfa081-b904-410a-afe2-0a2b704d9297 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:40.287235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1036/2058] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:52:40 2022] PUT /identity/v3/projects/957eeec9fcb7469d92262dd5446f6986/users/44bfdeefdba14289a7be2e9356346f34/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:40.297895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41b9f6-005a-4742-ba48-3eb9a43dd6a7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:40.298338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41b9f6-005a-4742-ba48-3eb9a43dd6a7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:40.298515 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41b9f6-005a-4742-ba48-3eb9a43dd6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:40.309399 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:40.352469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8d41b9f6-005a-4742-ba48-3eb9a43dd6a7 None None] MFA Rules not processed for user `44bfdeefdba14289a7be2e9356346f34`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:40.354051 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8d41b9f6-005a-4742-ba48-3eb9a43dd6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:40.566015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1023/2059] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:40.571282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfa081-b904-410a-afe2-0a2b704d9297 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:40.728798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfa081-b904-410a-afe2-0a2b704d9297 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:40.743090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace990b9-01f3-4899-8a70-d3e484eb8afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:40.743737 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace990b9-01f3-4899-8a70-d3e484eb8afa admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:40.744038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace990b9-01f3-4899-8a70-d3e484eb8afa admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:40.746499 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace990b9-01f3-4899-8a70-d3e484eb8afa admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:40.749681 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace990b9-01f3-4899-8a70-d3e484eb8afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:40.752383 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace990b9-01f3-4899-8a70-d3e484eb8afa admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:40.803636 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:40.821855 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1037/2060] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:52:40 2022] POST /identity/v3/projects => generated 375 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:52:40.827946 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41b9f6-005a-4742-ba48-3eb9a43dd6a7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:40.842616 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41b9f6-005a-4742-ba48-3eb9a43dd6a7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:41.009828 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86598591-617b-4e55-b1c6-53db6d7648f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:41.011058 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86598591-617b-4e55-b1c6-53db6d7648f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:41.011511 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86598591-617b-4e55-b1c6-53db6d7648f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:41.012765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86598591-617b-4e55-b1c6-53db6d7648f7 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:41.014111 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86598591-617b-4e55-b1c6-53db6d7648f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:41.032799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86598591-617b-4e55-b1c6-53db6d7648f7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:41.079519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace990b9-01f3-4899-8a70-d3e484eb8afa admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:41.083154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1024/2061] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:52:40 2022] POST /identity/v3/users => generated 294 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:52:41.093397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace990b9-01f3-4899-8a70-d3e484eb8afa admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:41.109605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d56b3-1c27-43ae-94b7-8830ed8b3667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:41.110258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d56b3-1c27-43ae-94b7-8830ed8b3667 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:41.110620 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d56b3-1c27-43ae-94b7-8830ed8b3667 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:41.110895 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d56b3-1c27-43ae-94b7-8830ed8b3667 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:41.111902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d56b3-1c27-43ae-94b7-8830ed8b3667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:41.113793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d56b3-1c27-43ae-94b7-8830ed8b3667 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:41.126678 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1038/2062] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:52:41.131219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86598591-617b-4e55-b1c6-53db6d7648f7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:41.146728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86598591-617b-4e55-b1c6-53db6d7648f7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:41.162287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e4535-3d70-43c9-bf0a-1e70d9ea0e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:41.163185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e4535-3d70-43c9-bf0a-1e70d9ea0e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:41.169124 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e4535-3d70-43c9-bf0a-1e70d9ea0e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:41.169442 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e4535-3d70-43c9-bf0a-1e70d9ea0e48 admin admin] PATH_INFO: `/v3/projects/90c53b25ed134d9ca88cb6718316a59c/users/187785e300c04ab686902ff61c78430f/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:41.185781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e4535-3d70-43c9-bf0a-1e70d9ea0e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c53b25ed134d9ca88cb6718316a59c, user_id=187785e300c04ab686902ff61c78430f, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:41.188869 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e4535-3d70-43c9-bf0a-1e70d9ea0e48 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:41.205282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1025/2063] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:52:41 2022] PUT /identity/v3/projects/90c53b25ed134d9ca88cb6718316a59c/users/187785e300c04ab686902ff61c78430f/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:41.209364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d56b3-1c27-43ae-94b7-8830ed8b3667 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:41.226364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d56b3-1c27-43ae-94b7-8830ed8b3667 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:41.304328 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5b3a7-a893-4fb4-af95-e163ba8e44a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:41.304959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5b3a7-a893-4fb4-af95-e163ba8e44a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:41.305233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5b3a7-a893-4fb4-af95-e163ba8e44a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:41.305496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5b3a7-a893-4fb4-af95-e163ba8e44a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:41.306137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5b3a7-a893-4fb4-af95-e163ba8e44a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:41.307739 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5b3a7-a893-4fb4-af95-e163ba8e44a1 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:41.318695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1039/2064] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:52:41.322926 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e4535-3d70-43c9-bf0a-1e70d9ea0e48 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:41.334938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e4535-3d70-43c9-bf0a-1e70d9ea0e48 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:41.341794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd561f73-4030-4ddb-a486-4abbf4c705e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:41.342561 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561f73-4030-4ddb-a486-4abbf4c705e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:41.342901 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561f73-4030-4ddb-a486-4abbf4c705e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:41.343307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561f73-4030-4ddb-a486-4abbf4c705e2 admin admin] PATH_INFO: `/v3/projects/90c53b25ed134d9ca88cb6718316a59c/users/187785e300c04ab686902ff61c78430f/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:41.345338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd561f73-4030-4ddb-a486-4abbf4c705e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c53b25ed134d9ca88cb6718316a59c, user_id=187785e300c04ab686902ff61c78430f, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:41.348230 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd561f73-4030-4ddb-a486-4abbf4c705e2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:41.367641 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1026/2065] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:52:41 2022] PUT /identity/v3/projects/90c53b25ed134d9ca88cb6718316a59c/users/187785e300c04ab686902ff61c78430f/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:52:41.373164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5fffd-d2de-483d-a58d-931e57143626 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:41.373423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5fffd-d2de-483d-a58d-931e57143626 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:41.373700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5fffd-d2de-483d-a58d-931e57143626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:41.406964 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-42b5fffd-d2de-483d-a58d-931e57143626 None None] MFA Rules not processed for user `187785e300c04ab686902ff61c78430f`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:41.408472 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-42b5fffd-d2de-483d-a58d-931e57143626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:41.501515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1040/2066] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:41.578943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd561f73-4030-4ddb-a486-4abbf4c705e2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:41.581562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dd561f73-4030-4ddb-a486-4abbf4c705e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:41.687215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:41.694404 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd561f73-4030-4ddb-a486-4abbf4c705e2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:41.699607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b0114-7141-43f9-86ba-c2a62b7c419f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:41.700592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b0114-7141-43f9-86ba-c2a62b7c419f service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:41.701012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b0114-7141-43f9-86ba-c2a62b7c419f service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:41.701423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b0114-7141-43f9-86ba-c2a62b7c419f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:41.802879 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b0114-7141-43f9-86ba-c2a62b7c419f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:41.804442 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b0114-7141-43f9-86ba-c2a62b7c419f service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:41.815917 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1027/2067] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:52:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:52:49.930458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5fffd-d2de-483d-a58d-931e57143626 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:49.932662 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-42b5fffd-d2de-483d-a58d-931e57143626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:49.954577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5fffd-d2de-483d-a58d-931e57143626 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:49.963570 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c1ecd-9a40-48ea-b8ef-83e207296892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:49.965338 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c1ecd-9a40-48ea-b8ef-83e207296892 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:49.968196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c1ecd-9a40-48ea-b8ef-83e207296892 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:49.968753 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c1ecd-9a40-48ea-b8ef-83e207296892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:50.092371 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c1ecd-9a40-48ea-b8ef-83e207296892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:50.094045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c1ecd-9a40-48ea-b8ef-83e207296892 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:50.111543 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1041/2068] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:52:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:52:50.546213 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b0114-7141-43f9-86ba-c2a62b7c419f service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:50.548182 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5e5b0114-7141-43f9-86ba-c2a62b7c419f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:50.692981 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b0114-7141-43f9-86ba-c2a62b7c419f service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:50.698315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44000f98-4156-48f5-bab3-1aa9d7fbfa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:50.699292 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44000f98-4156-48f5-bab3-1aa9d7fbfa09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:50.699785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44000f98-4156-48f5-bab3-1aa9d7fbfa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:50.700370 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44000f98-4156-48f5-bab3-1aa9d7fbfa09 admin admin] PATH_INFO: `/v3/users/da7b3e62c94f4e499522eab50119df9e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:50.701760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44000f98-4156-48f5-bab3-1aa9d7fbfa09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da7b3e62c94f4e499522eab50119df9e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:50.704425 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44000f98-4156-48f5-bab3-1aa9d7fbfa09 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:50.773566 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44000f98-4156-48f5-bab3-1aa9d7fbfa09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da7b3e62c94f4e499522eab50119df9e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:50.773951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44000f98-4156-48f5-bab3-1aa9d7fbfa09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da7b3e62c94f4e499522eab50119df9e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:50.776299 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44000f98-4156-48f5-bab3-1aa9d7fbfa09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da7b3e62c94f4e499522eab50119df9e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:50.799851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44000f98-4156-48f5-bab3-1aa9d7fbfa09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da7b3e62c94f4e499522eab50119df9e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:50.811294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44000f98-4156-48f5-bab3-1aa9d7fbfa09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da7b3e62c94f4e499522eab50119df9e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:50.835161 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1028/2069] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:52:50 2022] DELETE /identity/v3/users/da7b3e62c94f4e499522eab50119df9e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:50.840574 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54350bb-3da4-4a65-becd-37fd19e9121e None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:50.840980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54350bb-3da4-4a65-becd-37fd19e9121e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:50.841399 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54350bb-3da4-4a65-becd-37fd19e9121e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:50.852579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:50.867410 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-f54350bb-3da4-4a65-becd-37fd19e9121e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:50.868759 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f54350bb-3da4-4a65-becd-37fd19e9121e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:50.959496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1042/2070] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:50.969779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44000f98-4156-48f5-bab3-1aa9d7fbfa09 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:50.971226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-44000f98-4156-48f5-bab3-1aa9d7fbfa09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:51.002162 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:51.055645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44000f98-4156-48f5-bab3-1aa9d7fbfa09 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:51.060173 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a07dc9-4f05-4130-bd39-653e6776bbd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:51.061000 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a07dc9-4f05-4130-bd39-653e6776bbd9 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:51.061281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a07dc9-4f05-4130-bd39-653e6776bbd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:51.061543 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a07dc9-4f05-4130-bd39-653e6776bbd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:51.075445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a07dc9-4f05-4130-bd39-653e6776bbd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:51.075445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a07dc9-4f05-4130-bd39-653e6776bbd9 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:51.086980 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1029/2071] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:52:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:52:51.104594 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54350bb-3da4-4a65-becd-37fd19e9121e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:51.106132 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f54350bb-3da4-4a65-becd-37fd19e9121e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:51.183157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54350bb-3da4-4a65-becd-37fd19e9121e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:51.189160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6703b21-bf92-49a3-8ed2-7dd808733489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:51.189854 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6703b21-bf92-49a3-8ed2-7dd808733489 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:51.190065 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6703b21-bf92-49a3-8ed2-7dd808733489 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:51.190331 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6703b21-bf92-49a3-8ed2-7dd808733489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:51.192403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e6703b21-bf92-49a3-8ed2-7dd808733489 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:51.273628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6703b21-bf92-49a3-8ed2-7dd808733489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:51.275103 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6703b21-bf92-49a3-8ed2-7dd808733489 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:51.292911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1043/2072] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:52:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:52:51.489976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a07dc9-4f05-4130-bd39-653e6776bbd9 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:51.491698 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-72a07dc9-4f05-4130-bd39-653e6776bbd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:51.517492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a07dc9-4f05-4130-bd39-653e6776bbd9 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:51.546333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557ba33-a5d6-4da8-b6a5-d89ddb099568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:51.547061 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557ba33-a5d6-4da8-b6a5-d89ddb099568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:51.547277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557ba33-a5d6-4da8-b6a5-d89ddb099568 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:51.547603 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557ba33-a5d6-4da8-b6a5-d89ddb099568 admin admin] PATH_INFO: `/v3/projects/c40b016a3d634d829eb43b9e6bed02a2` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:51.548989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557ba33-a5d6-4da8-b6a5-d89ddb099568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c40b016a3d634d829eb43b9e6bed02a2)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:51.557976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557ba33-a5d6-4da8-b6a5-d89ddb099568 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:51.705991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7557ba33-a5d6-4da8-b6a5-d89ddb099568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c40b016a3d634d829eb43b9e6bed02a2', 'request_id': 'req-7557ba33-a5d6-4da8-b6a5-d89ddb099568'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:51.716798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7557ba33-a5d6-4da8-b6a5-d89ddb099568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c40b016a3d634d829eb43b9e6bed02a2', 'request_id': 'req-7557ba33-a5d6-4da8-b6a5-d89ddb099568'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:51.735831 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7557ba33-a5d6-4da8-b6a5-d89ddb099568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c40b016a3d634d829eb43b9e6bed02a2', 'request_id': 'req-7557ba33-a5d6-4da8-b6a5-d89ddb099568'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:51.758001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7557ba33-a5d6-4da8-b6a5-d89ddb099568 admin admin] The token cache is being invalidate because project c40b016a3d634d829eb43b9e6bed02a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:52:51.758303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7557ba33-a5d6-4da8-b6a5-d89ddb099568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:51.764553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7557ba33-a5d6-4da8-b6a5-d89ddb099568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:51.766544 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1030/2073] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:52:51 2022] DELETE /identity/v3/projects/c40b016a3d634d829eb43b9e6bed02a2 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:51.771710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0345ef-34db-49ea-b5a2-5e7aecb8890c None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:51.772015 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0345ef-34db-49ea-b5a2-5e7aecb8890c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:51.772287 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0345ef-34db-49ea-b5a2-5e7aecb8890c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:51.800155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-0c0345ef-34db-49ea-b5a2-5e7aecb8890c None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:51.800155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0c0345ef-34db-49ea-b5a2-5e7aecb8890c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:51.865440 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:51.889837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1044/2074] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:52:51.893891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2677314-f8cf-4ddf-9fcf-5d7c00c45c3c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:51.894153 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2677314-f8cf-4ddf-9fcf-5d7c00c45c3c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:51.894450 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2677314-f8cf-4ddf-9fcf-5d7c00c45c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:51.917386 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c2677314-f8cf-4ddf-9fcf-5d7c00c45c3c None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:51.918690 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c2677314-f8cf-4ddf-9fcf-5d7c00c45c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:51.927946 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1031/2075] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:51.932479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0345ef-34db-49ea-b5a2-5e7aecb8890c None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:51.943464 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0345ef-34db-49ea-b5a2-5e7aecb8890c None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:51.948529 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845907d-0e65-46dd-918a-69f6fa442dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:51.949152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845907d-0e65-46dd-918a-69f6fa442dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:51.949565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845907d-0e65-46dd-918a-69f6fa442dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:51.949856 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845907d-0e65-46dd-918a-69f6fa442dc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:51.950540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845907d-0e65-46dd-918a-69f6fa442dc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:52:51.950859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845907d-0e65-46dd-918a-69f6fa442dc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:51.952477 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845907d-0e65-46dd-918a-69f6fa442dc4 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:51.968801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1045/2076] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:52:51.971977 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2677314-f8cf-4ddf-9fcf-5d7c00c45c3c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:51.991952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2677314-f8cf-4ddf-9fcf-5d7c00c45c3c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:51.997481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb60d9-72bd-4e2b-a60f-8e98694f065e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:51.998104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb60d9-72bd-4e2b-a60f-8e98694f065e admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:51.998376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb60d9-72bd-4e2b-a60f-8e98694f065e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:51.998643 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb60d9-72bd-4e2b-a60f-8e98694f065e admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:51.999277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb60d9-72bd-4e2b-a60f-8e98694f065e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:52.001260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb60d9-72bd-4e2b-a60f-8e98694f065e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:52.031035 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1032/2077] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:52:51 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:52:52.035647 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845907d-0e65-46dd-918a-69f6fa442dc4 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:52.045900 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845907d-0e65-46dd-918a-69f6fa442dc4 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:52.107922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f86036-6a34-4e0d-aaab-d6251dfd760f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:52.108569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f86036-6a34-4e0d-aaab-d6251dfd760f admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:52.108845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f86036-6a34-4e0d-aaab-d6251dfd760f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:52.109145 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f86036-6a34-4e0d-aaab-d6251dfd760f admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:52.109777 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f86036-6a34-4e0d-aaab-d6251dfd760f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:52.111848 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f86036-6a34-4e0d-aaab-d6251dfd760f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:52.136945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1046/2078] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:52:52 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:52:52.141391 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb60d9-72bd-4e2b-a60f-8e98694f065e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:52.153142 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb60d9-72bd-4e2b-a60f-8e98694f065e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:52.159447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a855b4d-202e-4b2a-8b18-0c2c12dc052f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:52.160085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a855b4d-202e-4b2a-8b18-0c2c12dc052f admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:52.160354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a855b4d-202e-4b2a-8b18-0c2c12dc052f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:52.160617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a855b4d-202e-4b2a-8b18-0c2c12dc052f admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:52.161256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a855b4d-202e-4b2a-8b18-0c2c12dc052f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:52.162864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a855b4d-202e-4b2a-8b18-0c2c12dc052f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:52.173826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1033/2079] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:52:52.177995 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f86036-6a34-4e0d-aaab-d6251dfd760f admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:52.192832 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f86036-6a34-4e0d-aaab-d6251dfd760f admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:52.199934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707a092-72d5-4ef2-9d8d-d34effa6c515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:52.200485 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707a092-72d5-4ef2-9d8d-d34effa6c515 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:52.200765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707a092-72d5-4ef2-9d8d-d34effa6c515 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:52.201034 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707a092-72d5-4ef2-9d8d-d34effa6c515 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:52.201688 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707a092-72d5-4ef2-9d8d-d34effa6c515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:52.203306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707a092-72d5-4ef2-9d8d-d34effa6c515 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:52.218699 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1047/2080] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:52:52.223469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a855b4d-202e-4b2a-8b18-0c2c12dc052f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:52.234569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a855b4d-202e-4b2a-8b18-0c2c12dc052f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:52.240536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ff9aa-1928-4062-a2e2-533e6d41d5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:52.241239 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ff9aa-1928-4062-a2e2-533e6d41d5b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:52.241513 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ff9aa-1928-4062-a2e2-533e6d41d5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:52.241780 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ff9aa-1928-4062-a2e2-533e6d41d5b7 admin admin] PATH_INFO: `/v3/projects/243c0b9b4bd742659061745c8dc4c696/users/193bf7a17f544770a27c07f6b8cc0988/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:52.258358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ff9aa-1928-4062-a2e2-533e6d41d5b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243c0b9b4bd742659061745c8dc4c696, user_id=193bf7a17f544770a27c07f6b8cc0988, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:52.261183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ff9aa-1928-4062-a2e2-533e6d41d5b7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:52.277824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1034/2081] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:52:52 2022] PUT /identity/v3/projects/243c0b9b4bd742659061745c8dc4c696/users/193bf7a17f544770a27c07f6b8cc0988/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:52.283077 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65f516-8740-4886-936d-1418a9769c60 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:52.283364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65f516-8740-4886-936d-1418a9769c60 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:52.283638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65f516-8740-4886-936d-1418a9769c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:52.303376 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-dc65f516-8740-4886-936d-1418a9769c60 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:52.304673 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-dc65f516-8740-4886-936d-1418a9769c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:52.381861 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1048/2082] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:52.395498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ff9aa-1928-4062-a2e2-533e6d41d5b7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:52.398210 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-583ff9aa-1928-4062-a2e2-533e6d41d5b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:52.457014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:52.472667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ff9aa-1928-4062-a2e2-533e6d41d5b7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:52.477227 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a01bd-9140-4308-baec-95c0760385ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:52.477835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a01bd-9140-4308-baec-95c0760385ad service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:52.478106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a01bd-9140-4308-baec-95c0760385ad service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:52.478729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a01bd-9140-4308-baec-95c0760385ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:52.488412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a01bd-9140-4308-baec-95c0760385ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:52.489847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a01bd-9140-4308-baec-95c0760385ad service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:52.499604 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1035/2083] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:52:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:52:55.569938 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65f516-8740-4886-936d-1418a9769c60 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:55.569938 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-dc65f516-8740-4886-936d-1418a9769c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:55.584777 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65f516-8740-4886-936d-1418a9769c60 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:55.590513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d37994-b162-4535-b02f-b1feedcba8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:55.591266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d37994-b162-4535-b02f-b1feedcba8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:55.591608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d37994-b162-4535-b02f-b1feedcba8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:55.591977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d37994-b162-4535-b02f-b1feedcba8ff admin admin] PATH_INFO: `/v3/users/56b68e26810b4329901a45db6a23d262` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:55.594020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d37994-b162-4535-b02f-b1feedcba8ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b68e26810b4329901a45db6a23d262)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:55.596157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d37994-b162-4535-b02f-b1feedcba8ff admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:55.662417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-15d37994-b162-4535-b02f-b1feedcba8ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b68e26810b4329901a45db6a23d262'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:55.671202 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-15d37994-b162-4535-b02f-b1feedcba8ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b68e26810b4329901a45db6a23d262'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:55.672083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-15d37994-b162-4535-b02f-b1feedcba8ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b68e26810b4329901a45db6a23d262'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:55.672955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-15d37994-b162-4535-b02f-b1feedcba8ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b68e26810b4329901a45db6a23d262'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:55.675660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:55.691653 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-15d37994-b162-4535-b02f-b1feedcba8ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b68e26810b4329901a45db6a23d262'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:55.723462 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1049/2084] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:52:55 2022] DELETE /identity/v3/users/56b68e26810b4329901a45db6a23d262 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:55.729177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fca64-b50b-4a72-8924-71046348becb None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:55.729729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fca64-b50b-4a72-8924-71046348becb None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:55.729827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fca64-b50b-4a72-8924-71046348becb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:55.794473 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d61fca64-b50b-4a72-8924-71046348becb None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:55.802344 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d61fca64-b50b-4a72-8924-71046348becb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:55.967692 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1036/2085] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:56.021512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d37994-b162-4535-b02f-b1feedcba8ff admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:56.022507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-15d37994-b162-4535-b02f-b1feedcba8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:56.137406 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d37994-b162-4535-b02f-b1feedcba8ff admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:56.141776 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5053d-9e62-4bf4-8b68-a6e75592dc9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:56.142948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5053d-9e62-4bf4-8b68-a6e75592dc9e service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:56.142948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5053d-9e62-4bf4-8b68-a6e75592dc9e service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:56.143797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5053d-9e62-4bf4-8b68-a6e75592dc9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:56.157966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5053d-9e62-4bf4-8b68-a6e75592dc9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:56.159667 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5053d-9e62-4bf4-8b68-a6e75592dc9e service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:56.175323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1050/2086] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:52:57.163290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61fca64-b50b-4a72-8924-71046348becb None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:57.164954 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d61fca64-b50b-4a72-8924-71046348becb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:57.186486 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61fca64-b50b-4a72-8924-71046348becb None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:57.197962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:57.199132 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:57.200226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:57.200831 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] PATH_INFO: `/v3/projects/e39808ea214944b6ab06774fa1b501e7` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:57.202468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e39808ea214944b6ab06774fa1b501e7)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:57.204853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:57.335851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e39808ea214944b6ab06774fa1b501e7', 'request_id': 'req-4354033e-f365-4779-8f99-827021ea16c2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:57.351039 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e39808ea214944b6ab06774fa1b501e7', 'request_id': 'req-4354033e-f365-4779-8f99-827021ea16c2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:57.376992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e39808ea214944b6ab06774fa1b501e7', 'request_id': 'req-4354033e-f365-4779-8f99-827021ea16c2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:57.396367 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] The token cache is being invalidate because project e39808ea214944b6ab06774fa1b501e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:52:57.396997 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:57.397728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:57.399450 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1037/2087] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:52:57 2022] DELETE /identity/v3/projects/e39808ea214944b6ab06774fa1b501e7 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:57.404898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5053d-9e62-4bf4-8b68-a6e75592dc9e service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:57.423412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5053d-9e62-4bf4-8b68-a6e75592dc9e service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:57.540168 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2c9fd-3950-409b-918a-1488741a1a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:57.540617 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2c9fd-3950-409b-918a-1488741a1a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:57.541061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2c9fd-3950-409b-918a-1488741a1a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:57.541339 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2c9fd-3950-409b-918a-1488741a1a53 admin admin] PATH_INFO: `/v3/users/44bfdeefdba14289a7be2e9356346f34` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:57.542736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2c9fd-3950-409b-918a-1488741a1a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44bfdeefdba14289a7be2e9356346f34)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:57.544878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2c9fd-3950-409b-918a-1488741a1a53 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:57.670204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c6e2c9fd-3950-409b-918a-1488741a1a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44bfdeefdba14289a7be2e9356346f34'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:57.731288 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c6e2c9fd-3950-409b-918a-1488741a1a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44bfdeefdba14289a7be2e9356346f34'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:57.732104 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c6e2c9fd-3950-409b-918a-1488741a1a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44bfdeefdba14289a7be2e9356346f34'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:57.732714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c6e2c9fd-3950-409b-918a-1488741a1a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44bfdeefdba14289a7be2e9356346f34'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:57.760755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c6e2c9fd-3950-409b-918a-1488741a1a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44bfdeefdba14289a7be2e9356346f34'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:57.794763 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:57.809443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1051/2088] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:52:57 2022] DELETE /identity/v3/users/44bfdeefdba14289a7be2e9356346f34 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:58.426161 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:58.428255 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:58.450760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354033e-f365-4779-8f99-827021ea16c2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:58.635706 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc456899-9f09-4b3d-9388-7f49a71e5691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:58.636465 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc456899-9f09-4b3d-9388-7f49a71e5691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:58.636745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc456899-9f09-4b3d-9388-7f49a71e5691 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:58.637090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc456899-9f09-4b3d-9388-7f49a71e5691 admin admin] PATH_INFO: `/v3/projects/957eeec9fcb7469d92262dd5446f6986` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:58.638494 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc456899-9f09-4b3d-9388-7f49a71e5691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=957eeec9fcb7469d92262dd5446f6986)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:58.640869 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc456899-9f09-4b3d-9388-7f49a71e5691 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:58.668953 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:58.783897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bc456899-9f09-4b3d-9388-7f49a71e5691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '957eeec9fcb7469d92262dd5446f6986', 'request_id': 'req-bc456899-9f09-4b3d-9388-7f49a71e5691'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:58.800286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bc456899-9f09-4b3d-9388-7f49a71e5691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '957eeec9fcb7469d92262dd5446f6986', 'request_id': 'req-bc456899-9f09-4b3d-9388-7f49a71e5691'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:58.816883 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bc456899-9f09-4b3d-9388-7f49a71e5691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '957eeec9fcb7469d92262dd5446f6986', 'request_id': 'req-bc456899-9f09-4b3d-9388-7f49a71e5691'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:58.840086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bc456899-9f09-4b3d-9388-7f49a71e5691 admin admin] The token cache is being invalidate because project 957eeec9fcb7469d92262dd5446f6986 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:52:58.840774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bc456899-9f09-4b3d-9388-7f49a71e5691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:58.842167 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bc456899-9f09-4b3d-9388-7f49a71e5691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:58.848889 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1038/2089] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:52:58 2022] DELETE /identity/v3/projects/957eeec9fcb7469d92262dd5446f6986 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:52:58.850191 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2c9fd-3950-409b-918a-1488741a1a53 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:58.852282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c6e2c9fd-3950-409b-918a-1488741a1a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:58.896929 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2c9fd-3950-409b-918a-1488741a1a53 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:59.124748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa72a1-dd69-4828-8fdc-fba791df2251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:59.128826 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa72a1-dd69-4828-8fdc-fba791df2251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:59.129143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa72a1-dd69-4828-8fdc-fba791df2251 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:59.129449 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa72a1-dd69-4828-8fdc-fba791df2251 admin admin] PATH_INFO: `/v3/users/187785e300c04ab686902ff61c78430f` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:59.134413 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa72a1-dd69-4828-8fdc-fba791df2251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=187785e300c04ab686902ff61c78430f)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:59.140630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa72a1-dd69-4828-8fdc-fba791df2251 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:59.275545 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-13fa72a1-dd69-4828-8fdc-fba791df2251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '187785e300c04ab686902ff61c78430f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:59.288779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-13fa72a1-dd69-4828-8fdc-fba791df2251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '187785e300c04ab686902ff61c78430f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:59.291109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-13fa72a1-dd69-4828-8fdc-fba791df2251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '187785e300c04ab686902ff61c78430f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:59.292748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-13fa72a1-dd69-4828-8fdc-fba791df2251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '187785e300c04ab686902ff61c78430f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:59.316605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-13fa72a1-dd69-4828-8fdc-fba791df2251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '187785e300c04ab686902ff61c78430f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:52:59.348629 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1052/2090] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:52:58 2022] DELETE /identity/v3/users/187785e300c04ab686902ff61c78430f => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:52:59.425663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817475bf-8ff9-4139-9a2f-3c63cda49720 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:59.425940 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817475bf-8ff9-4139-9a2f-3c63cda49720 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:59.426211 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817475bf-8ff9-4139-9a2f-3c63cda49720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:59.467716 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-817475bf-8ff9-4139-9a2f-3c63cda49720 None None] MFA Rules not processed for user `193bf7a17f544770a27c07f6b8cc0988`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:52:59.473114 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-817475bf-8ff9-4139-9a2f-3c63cda49720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:59.583916 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1039/2091] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:52:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:52:59.584228 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa72a1-dd69-4828-8fdc-fba791df2251 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:59.591358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-13fa72a1-dd69-4828-8fdc-fba791df2251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:59.767426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa72a1-dd69-4828-8fdc-fba791df2251 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:59.774649 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ead0f-3ce7-441c-bfde-b1a609553d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:59.775426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ead0f-3ce7-441c-bfde-b1a609553d9f admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:59.775703 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ead0f-3ce7-441c-bfde-b1a609553d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:59.776089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ead0f-3ce7-441c-bfde-b1a609553d9f admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:59.777403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ead0f-3ce7-441c-bfde-b1a609553d9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:59.781590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ead0f-3ce7-441c-bfde-b1a609553d9f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:52:59.824757 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:52:59.853335 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1053/2092] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:52:59 2022] POST /identity/v3/projects => generated 379 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:52:59.856380 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817475bf-8ff9-4139-9a2f-3c63cda49720 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:59.882971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817475bf-8ff9-4139-9a2f-3c63cda49720 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:52:59.983707 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ead0f-3ce7-441c-bfde-b1a609553d9f admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:52:59.985366 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7c59a-061b-491c-8520-6a4e981cdc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:52:59.985711 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-bd2ead0f-3ce7-441c-bfde-b1a609553d9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:52:59.986202 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7c59a-061b-491c-8520-6a4e981cdc80 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:52:59.986573 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7c59a-061b-491c-8520-6a4e981cdc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:52:59.987041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7c59a-061b-491c-8520-6a4e981cdc80 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:52:59.987711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7c59a-061b-491c-8520-6a4e981cdc80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:52:59.989856 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7c59a-061b-491c-8520-6a4e981cdc80 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:00.012185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ead0f-3ce7-441c-bfde-b1a609553d9f admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:00.021937 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5c1db-c0f0-4675-9a67-04cda1f80c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:00.023028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5c1db-c0f0-4675-9a67-04cda1f80c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:00.023504 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5c1db-c0f0-4675-9a67-04cda1f80c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:00.024624 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5c1db-c0f0-4675-9a67-04cda1f80c27 admin admin] PATH_INFO: `/v3/projects/90c53b25ed134d9ca88cb6718316a59c` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:00.026995 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5c1db-c0f0-4675-9a67-04cda1f80c27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c53b25ed134d9ca88cb6718316a59c)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:00.031127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5c1db-c0f0-4675-9a67-04cda1f80c27 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:00.036510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1040/2093] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:52:59 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:53:00.041594 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7c59a-061b-491c-8520-6a4e981cdc80 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:00.067856 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7c59a-061b-491c-8520-6a4e981cdc80 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:00.075662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c6675-0253-4597-a8c3-901e6c968bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:00.076814 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c6675-0253-4597-a8c3-901e6c968bfd admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:00.077327 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c6675-0253-4597-a8c3-901e6c968bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:00.077803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c6675-0253-4597-a8c3-901e6c968bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:00.079690 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c6675-0253-4597-a8c3-901e6c968bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:00.083126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c6675-0253-4597-a8c3-901e6c968bfd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:00.109735 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1041/2094] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:53:00.111329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c6675-0253-4597-a8c3-901e6c968bfd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:00.125360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c6675-0253-4597-a8c3-901e6c968bfd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:00.131577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c390ae-000b-4fb0-b60b-0c5086591eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:00.132349 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c390ae-000b-4fb0-b60b-0c5086591eec admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:00.132627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c390ae-000b-4fb0-b60b-0c5086591eec admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:00.132905 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c390ae-000b-4fb0-b60b-0c5086591eec admin admin] PATH_INFO: `/v3/projects/52d0343fa71c4c179a2178b88e6def9c/users/d36a68a6a2c240988aaa151bfe43fe1e/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:00.135912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:53:00.160761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c390ae-000b-4fb0-b60b-0c5086591eec admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d0343fa71c4c179a2178b88e6def9c, user_id=d36a68a6a2c240988aaa151bfe43fe1e, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:00.164193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c390ae-000b-4fb0-b60b-0c5086591eec admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:00.185142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-91f5c1db-c0f0-4675-9a67-04cda1f80c27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c53b25ed134d9ca88cb6718316a59c', 'request_id': 'req-91f5c1db-c0f0-4675-9a67-04cda1f80c27'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:53:00.187360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1042/2095] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:53:00 2022] PUT /identity/v3/projects/52d0343fa71c4c179a2178b88e6def9c/users/d36a68a6a2c240988aaa151bfe43fe1e/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:53:00.192402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c390ae-000b-4fb0-b60b-0c5086591eec admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:00.204402 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-91f5c1db-c0f0-4675-9a67-04cda1f80c27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c53b25ed134d9ca88cb6718316a59c', 'request_id': 'req-91f5c1db-c0f0-4675-9a67-04cda1f80c27'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:53:00.223258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-91f5c1db-c0f0-4675-9a67-04cda1f80c27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c53b25ed134d9ca88cb6718316a59c', 'request_id': 'req-91f5c1db-c0f0-4675-9a67-04cda1f80c27'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:53:00.245967 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-91f5c1db-c0f0-4675-9a67-04cda1f80c27 admin admin] The token cache is being invalidate because project 90c53b25ed134d9ca88cb6718316a59c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:53:00.246191 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-91f5c1db-c0f0-4675-9a67-04cda1f80c27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:53:00.246979 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-91f5c1db-c0f0-4675-9a67-04cda1f80c27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:53:00.253093 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1054/2096] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:52:59 2022] DELETE /identity/v3/projects/90c53b25ed134d9ca88cb6718316a59c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:53:00.263934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550966c4-4226-4889-8d0e-c70dd628e507 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:00.267023 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550966c4-4226-4889-8d0e-c70dd628e507 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:00.269247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550966c4-4226-4889-8d0e-c70dd628e507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:00.302022 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c390ae-000b-4fb0-b60b-0c5086591eec admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:00.310512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbacbf7-4c06-40dc-91ab-cc33cab807b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:00.311227 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbacbf7-4c06-40dc-91ab-cc33cab807b2 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:00.311506 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbacbf7-4c06-40dc-91ab-cc33cab807b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:00.311779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbacbf7-4c06-40dc-91ab-cc33cab807b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:00.312504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbacbf7-4c06-40dc-91ab-cc33cab807b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:00.315789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbacbf7-4c06-40dc-91ab-cc33cab807b2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:00.330879 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-550966c4-4226-4889-8d0e-c70dd628e507 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:53:00.332139 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-550966c4-4226-4889-8d0e-c70dd628e507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:00.335437 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1043/2097] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:53:00.336517 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbacbf7-4c06-40dc-91ab-cc33cab807b2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:00.338591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1bbacbf7-4c06-40dc-91ab-cc33cab807b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:00.362581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1055/2098] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:53:00.365029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc194f-8905-481c-835d-222047e6815e None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:00.365029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc194f-8905-481c-835d-222047e6815e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:00.365029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc194f-8905-481c-835d-222047e6815e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:00.375759 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbacbf7-4c06-40dc-91ab-cc33cab807b2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:00.385138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e2cb1-3363-4d60-b275-a0c1e564124d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:00.386869 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e2cb1-3363-4d60-b275-a0c1e564124d admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:00.388514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e2cb1-3363-4d60-b275-a0c1e564124d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:00.388826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e2cb1-3363-4d60-b275-a0c1e564124d admin admin] PATH_INFO: `/v3/projects/52d0343fa71c4c179a2178b88e6def9c/users/d36a68a6a2c240988aaa151bfe43fe1e/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:00.391510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e2cb1-3363-4d60-b275-a0c1e564124d admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d0343fa71c4c179a2178b88e6def9c, user_id=d36a68a6a2c240988aaa151bfe43fe1e, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:00.395717 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e2cb1-3363-4d60-b275-a0c1e564124d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:00.401180 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-c7cc194f-8905-481c-835d-222047e6815e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:53:00.403288 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c7cc194f-8905-481c-835d-222047e6815e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:00.417429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1044/2099] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:53:00 2022] PUT /identity/v3/projects/52d0343fa71c4c179a2178b88e6def9c/users/d36a68a6a2c240988aaa151bfe43fe1e/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:53:00.578721 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1056/2100] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:53:00.589106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e2cb1-3363-4d60-b275-a0c1e564124d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:00.625238 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e2cb1-3363-4d60-b275-a0c1e564124d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:00.642666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149083c-92c2-41d1-bfbf-8b24647e08fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:00.646265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149083c-92c2-41d1-bfbf-8b24647e08fb admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:00.646797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149083c-92c2-41d1-bfbf-8b24647e08fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:00.647263 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149083c-92c2-41d1-bfbf-8b24647e08fb admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:00.649450 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149083c-92c2-41d1-bfbf-8b24647e08fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:53:00.650078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149083c-92c2-41d1-bfbf-8b24647e08fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:00.654734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149083c-92c2-41d1-bfbf-8b24647e08fb admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:00.711583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc194f-8905-481c-835d-222047e6815e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:00.720564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1045/2101] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:53:00.733117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc194f-8905-481c-835d-222047e6815e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:00.743297 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645b7e7-e236-46dd-9251-8fcc14238ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:00.744564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645b7e7-e236-46dd-9251-8fcc14238ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:00.745141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645b7e7-e236-46dd-9251-8fcc14238ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:00.745742 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645b7e7-e236-46dd-9251-8fcc14238ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:00.747017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645b7e7-e236-46dd-9251-8fcc14238ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:00.752596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645b7e7-e236-46dd-9251-8fcc14238ca5 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:00.825546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149083c-92c2-41d1-bfbf-8b24647e08fb admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:00.827012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1057/2102] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:53:00 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:53:00.856788 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149083c-92c2-41d1-bfbf-8b24647e08fb admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:00.992222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd506c-dbe8-4637-8fcf-e8c022bbdbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:00.993070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd506c-dbe8-4637-8fcf-e8c022bbdbfd admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:00.994003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd506c-dbe8-4637-8fcf-e8c022bbdbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:00.994260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd506c-dbe8-4637-8fcf-e8c022bbdbfd admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:00.994833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd506c-dbe8-4637-8fcf-e8c022bbdbfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:00.999671 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd506c-dbe8-4637-8fcf-e8c022bbdbfd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:01.047606 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1046/2103] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:53:00 2022] POST /identity/v3/users => generated 292 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:53:01.053376 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645b7e7-e236-46dd-9251-8fcc14238ca5 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:01.071825 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645b7e7-e236-46dd-9251-8fcc14238ca5 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:01.082828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dbac6-43a3-42ff-83bd-ca5c0ef172c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:01.083950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dbac6-43a3-42ff-83bd-ca5c0ef172c4 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:01.084448 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dbac6-43a3-42ff-83bd-ca5c0ef172c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:01.084884 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dbac6-43a3-42ff-83bd-ca5c0ef172c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:01.086351 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00dbac6-43a3-42ff-83bd-ca5c0ef172c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:01.092305 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00dbac6-43a3-42ff-83bd-ca5c0ef172c4 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:01.104542 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1058/2104] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:53:01.109630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd506c-dbe8-4637-8fcf-e8c022bbdbfd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:01.152481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd506c-dbe8-4637-8fcf-e8c022bbdbfd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:01.171852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbac485-0d8b-4235-b4cb-7284a7c84a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:01.173205 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbac485-0d8b-4235-b4cb-7284a7c84a0d admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:01.173752 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbac485-0d8b-4235-b4cb-7284a7c84a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:01.174243 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbac485-0d8b-4235-b4cb-7284a7c84a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:01.175164 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbac485-0d8b-4235-b4cb-7284a7c84a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:01.179682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbac485-0d8b-4235-b4cb-7284a7c84a0d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:01.211368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1047/2105] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:53:01.214547 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dbac6-43a3-42ff-83bd-ca5c0ef172c4 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:01.249260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dbac6-43a3-42ff-83bd-ca5c0ef172c4 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:01.263446 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b130cd-2744-415a-a5b7-90d871f99b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:01.264207 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b130cd-2744-415a-a5b7-90d871f99b58 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:01.264495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b130cd-2744-415a-a5b7-90d871f99b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:01.264815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b130cd-2744-415a-a5b7-90d871f99b58 admin admin] PATH_INFO: `/v3/projects/190914cc865b4003ab4b735d937e130d/users/707bd577eee14892990367f396270f7b/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:01.296358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b130cd-2744-415a-a5b7-90d871f99b58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=190914cc865b4003ab4b735d937e130d, user_id=707bd577eee14892990367f396270f7b, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:01.299511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b130cd-2744-415a-a5b7-90d871f99b58 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:01.320471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1059/2106] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:53:01 2022] PUT /identity/v3/projects/190914cc865b4003ab4b735d937e130d/users/707bd577eee14892990367f396270f7b/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:53:01.325673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f00e5e-cbe3-4f92-87ac-b388e7039e13 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:01.326052 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f00e5e-cbe3-4f92-87ac-b388e7039e13 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:01.326466 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f00e5e-cbe3-4f92-87ac-b388e7039e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:01.362455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d8f00e5e-cbe3-4f92-87ac-b388e7039e13 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:53:01.365737 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d8f00e5e-cbe3-4f92-87ac-b388e7039e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:01.428064 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:53:01.515976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1048/2107] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:53:01.531447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b130cd-2744-415a-a5b7-90d871f99b58 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:01.535613 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-22b130cd-2744-415a-a5b7-90d871f99b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:01.574267 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:53:01.666188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b130cd-2744-415a-a5b7-90d871f99b58 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:01.677846 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a36c7-39b7-4f08-b782-0be277fbdb2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:01.678670 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a36c7-39b7-4f08-b782-0be277fbdb2e service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:01.678950 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a36c7-39b7-4f08-b782-0be277fbdb2e service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:01.679219 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a36c7-39b7-4f08-b782-0be277fbdb2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:01.693599 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a36c7-39b7-4f08-b782-0be277fbdb2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:01.695349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a36c7-39b7-4f08-b782-0be277fbdb2e service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:01.707544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1060/2108] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:53:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:53:04.334512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6964d79-ed46-4016-9555-4307fe95b464 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:04.335034 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6964d79-ed46-4016-9555-4307fe95b464 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:04.335034 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6964d79-ed46-4016-9555-4307fe95b464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:04.360647 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b552b9-0a9a-469b-a4f4-8104591b1d38 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:04.363917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b552b9-0a9a-469b-a4f4-8104591b1d38 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:04.364768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b552b9-0a9a-469b-a4f4-8104591b1d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:04.407174 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f6964d79-ed46-4016-9555-4307fe95b464 None None] MFA Rules not processed for user `707bd577eee14892990367f396270f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:53:04.408874 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f6964d79-ed46-4016-9555-4307fe95b464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:04.420648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-99b552b9-0a9a-469b-a4f4-8104591b1d38 None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:53:04.422083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-99b552b9-0a9a-469b-a4f4-8104591b1d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:04.584508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1049/2109] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:53:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:53:04.589974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1061/2110] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:53:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:53:04.600984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b552b9-0a9a-469b-a4f4-8104591b1d38 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:04.601300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6964d79-ed46-4016-9555-4307fe95b464 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:04.604626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f6964d79-ed46-4016-9555-4307fe95b464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:04.610822 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-99b552b9-0a9a-469b-a4f4-8104591b1d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:04.737628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b552b9-0a9a-469b-a4f4-8104591b1d38 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:04.754070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0627a-e3ef-4b2e-b82c-5a8da7e21a42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:04.754724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0627a-e3ef-4b2e-b82c-5a8da7e21a42 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:04.755003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0627a-e3ef-4b2e-b82c-5a8da7e21a42 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:04.755278 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0627a-e3ef-4b2e-b82c-5a8da7e21a42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:04.767752 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0627a-e3ef-4b2e-b82c-5a8da7e21a42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:04.770075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0627a-e3ef-4b2e-b82c-5a8da7e21a42 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:04.785964 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6964d79-ed46-4016-9555-4307fe95b464 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:04.802174 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1062/2111] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:53:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:53:04.841431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395de55c-9bc8-45e7-90bc-abe19b559a31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:04.845485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395de55c-9bc8-45e7-90bc-abe19b559a31 service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:04.847564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395de55c-9bc8-45e7-90bc-abe19b559a31 service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:04.849663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395de55c-9bc8-45e7-90bc-abe19b559a31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:04.902897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395de55c-9bc8-45e7-90bc-abe19b559a31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:04.904777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395de55c-9bc8-45e7-90bc-abe19b559a31 service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:04.930103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1050/2112] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:53:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:53:06.699177 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69687a3e-2501-4d58-9632-57259a6018f8 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:06.699859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69687a3e-2501-4d58-9632-57259a6018f8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:06.700066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69687a3e-2501-4d58-9632-57259a6018f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:06.765251 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-69687a3e-2501-4d58-9632-57259a6018f8 None None] MFA Rules not processed for user `707bd577eee14892990367f396270f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:53:06.770124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-69687a3e-2501-4d58-9632-57259a6018f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:06.804118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1063/2113] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:53:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:53:10.176495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf75c3-6dbe-404d-9bb0-cb64bcefeecd None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:10.177666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf75c3-6dbe-404d-9bb0-cb64bcefeecd None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:10.178481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf75c3-6dbe-404d-9bb0-cb64bcefeecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:10.241536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-dadf75c3-6dbe-404d-9bb0-cb64bcefeecd None None] MFA Rules not processed for user `d36a68a6a2c240988aaa151bfe43fe1e`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:53:10.243252 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dadf75c3-6dbe-404d-9bb0-cb64bcefeecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:10.332461 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:53:10.444531 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1051/2114] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:53:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:53:10.536729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69687a3e-2501-4d58-9632-57259a6018f8 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:10.538478 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-69687a3e-2501-4d58-9632-57259a6018f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:10.716094 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:53:10.743190 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69687a3e-2501-4d58-9632-57259a6018f8 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:10.770324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb5c4c-ff63-41f8-8c28-edd97dfa1d1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:10.771427 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb5c4c-ff63-41f8-8c28-edd97dfa1d1a service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:10.771936 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb5c4c-ff63-41f8-8c28-edd97dfa1d1a service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:10.772514 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb5c4c-ff63-41f8-8c28-edd97dfa1d1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:10.789353 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b2cb5c4c-ff63-41f8-8c28-edd97dfa1d1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:10.848922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf75c3-6dbe-404d-9bb0-cb64bcefeecd None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:10.872167 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf75c3-6dbe-404d-9bb0-cb64bcefeecd None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:10.882545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2edc35-c307-4bdd-814c-f4718d7f6222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:10.883189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2edc35-c307-4bdd-814c-f4718d7f6222 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:10.883462 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2edc35-c307-4bdd-814c-f4718d7f6222 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:10.883723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2edc35-c307-4bdd-814c-f4718d7f6222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:10.890737 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ae2edc35-c307-4bdd-814c-f4718d7f6222 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:10.948506 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb5c4c-ff63-41f8-8c28-edd97dfa1d1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:10.950429 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb5c4c-ff63-41f8-8c28-edd97dfa1d1a service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:10.975590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1064/2115] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:53:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:53:11.096160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2edc35-c307-4bdd-814c-f4718d7f6222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:11.100221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2edc35-c307-4bdd-814c-f4718d7f6222 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:11.117388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1052/2116] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:53:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:53:20.958268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=27 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:53:21.110224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=41 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:53:25.348874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb5c4c-ff63-41f8-8c28-edd97dfa1d1a service nova] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:25.360891 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb5c4c-ff63-41f8-8c28-edd97dfa1d1a service nova] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:25.365130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e837d-11d2-4a30-93b5-65694ec03ae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:25.366045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e837d-11d2-4a30-93b5-65694ec03ae2 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:25.366387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e837d-11d2-4a30-93b5-65694ec03ae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:25.366723 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e837d-11d2-4a30-93b5-65694ec03ae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:25.369142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-830e837d-11d2-4a30-93b5-65694ec03ae2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:25.443160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e837d-11d2-4a30-93b5-65694ec03ae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:25.444585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e837d-11d2-4a30-93b5-65694ec03ae2 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:25.454720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1065/2117] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:53:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:53:29.822519 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babde30-d05d-4b75-a59a-6d6df85fff98 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:29.823147 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babde30-d05d-4b75-a59a-6d6df85fff98 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:29.823147 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babde30-d05d-4b75-a59a-6d6df85fff98 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:29.824804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1053/2118] 10.222.0.36 () {54 vars in 921 bytes} [Sat Oct 15 13:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:53:29.831994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93082ed-2535-46f2-bdcb-f46f34851e57 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:29.833754 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93082ed-2535-46f2-bdcb-f46f34851e57 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:29.834289 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93082ed-2535-46f2-bdcb-f46f34851e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:29.867841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-b93082ed-2535-46f2-bdcb-f46f34851e57 None None] MFA Rules not processed for user `2e13692c1e83413a909d99bb7db2e4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:53:29.869403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b93082ed-2535-46f2-bdcb-f46f34851e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:29.894223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1066/2119] 10.222.0.36 () {58 vars in 1018 bytes} [Sat Oct 15 13:53:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:53:29.919741 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babde30-d05d-4b75-a59a-6d6df85fff98 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:29.921593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0babde30-d05d-4b75-a59a-6d6df85fff98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:30.019802 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babde30-d05d-4b75-a59a-6d6df85fff98 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:30.027246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409f851-2a38-4a3f-9028-6d6c7ff56413 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f5aabcb1414413a8dce7b76ce701582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:30.029672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409f851-2a38-4a3f-9028-6d6c7ff56413 service glance] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:30.030139 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409f851-2a38-4a3f-9028-6d6c7ff56413 service glance] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:30.030751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409f851-2a38-4a3f-9028-6d6c7ff56413 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:30.046186 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409f851-2a38-4a3f-9028-6d6c7ff56413 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:30.047847 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409f851-2a38-4a3f-9028-6d6c7ff56413 service glance] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:30.073784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1054/2120] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:53:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:53:30.149024 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1100e8-ae1e-4246-b39d-b8356b618452 None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:30.149307 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1100e8-ae1e-4246-b39d-b8356b618452 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:30.149750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1100e8-ae1e-4246-b39d-b8356b618452 None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:30.151445 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1067/2121] 10.222.0.36 () {54 vars in 921 bytes} [Sat Oct 15 13:53:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:53:30.157113 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d01311-8e9e-4ead-b6d0-19027746ffd5 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:30.157697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d01311-8e9e-4ead-b6d0-19027746ffd5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:30.158148 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d01311-8e9e-4ead-b6d0-19027746ffd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:30.182508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-48d01311-8e9e-4ead-b6d0-19027746ffd5 None None] MFA Rules not processed for user `90985ec8c43a4d7895942731dc1d002c`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:53:30.184069 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-48d01311-8e9e-4ead-b6d0-19027746ffd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:30.191567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1055/2122] 10.222.0.36 () {58 vars in 1018 bytes} [Sat Oct 15 13:53:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:53:30.205549 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1100e8-ae1e-4246-b39d-b8356b618452 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:30.218883 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1100e8-ae1e-4246-b39d-b8356b618452 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:30.223070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd6065-b981-42fa-9f53-e6801b78ccd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:30.223923 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd6065-b981-42fa-9f53-e6801b78ccd1 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:30.224213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd6065-b981-42fa-9f53-e6801b78ccd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:30.224996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd6065-b981-42fa-9f53-e6801b78ccd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:30.234694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd6065-b981-42fa-9f53-e6801b78ccd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:30.236869 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd6065-b981-42fa-9f53-e6801b78ccd1 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:30.246866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1068/2123] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:53:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:53:30.351574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b5dc1-450a-4550-94b2-c31dc1a71cc0 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:30.351776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b5dc1-450a-4550-94b2-c31dc1a71cc0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:30.352136 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b5dc1-450a-4550-94b2-c31dc1a71cc0 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:30.355184 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1056/2124] 10.222.0.36 () {54 vars in 921 bytes} [Sat Oct 15 13:53:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:53:30.361564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052114b-b11a-449b-b4fe-e49099da0417 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:30.361838 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052114b-b11a-449b-b4fe-e49099da0417 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:30.362110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052114b-b11a-449b-b4fe-e49099da0417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:30.390393 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-5052114b-b11a-449b-b4fe-e49099da0417 None None] MFA Rules not processed for user `2e13692c1e83413a909d99bb7db2e4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:53:30.391841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-5052114b-b11a-449b-b4fe-e49099da0417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:30.402683 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1069/2125] 10.222.0.36 () {58 vars in 1018 bytes} [Sat Oct 15 13:53:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:53:30.415515 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b5dc1-450a-4550-94b2-c31dc1a71cc0 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:30.417522 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c66b5dc1-450a-4550-94b2-c31dc1a71cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:30.436678 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b5dc1-450a-4550-94b2-c31dc1a71cc0 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:30.441626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a56756-8954-4551-b0a1-005407657520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:30.442760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a56756-8954-4551-b0a1-005407657520 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:30.442760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a56756-8954-4551-b0a1-005407657520 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:30.442997 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a56756-8954-4551-b0a1-005407657520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:30.454765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a56756-8954-4551-b0a1-005407657520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:30.456270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a56756-8954-4551-b0a1-005407657520 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:30.468228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1057/2126] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:53:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:53:32.370596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052114b-b11a-449b-b4fe-e49099da0417 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:32.381293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052114b-b11a-449b-b4fe-e49099da0417 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:32.385330 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f1e0a-4b62-48e4-a062-4be205b534a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:32.385982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f1e0a-4b62-48e4-a062-4be205b534a1 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:32.386260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f1e0a-4b62-48e4-a062-4be205b534a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:32.386621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f1e0a-4b62-48e4-a062-4be205b534a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:32.388461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2c0f1e0a-4b62-48e4-a062-4be205b534a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:32.402596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f1e0a-4b62-48e4-a062-4be205b534a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:32.404115 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f1e0a-4b62-48e4-a062-4be205b534a1 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:32.418459 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1070/2127] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:53:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:53:33.641414 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a56756-8954-4551-b0a1-005407657520 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:33.651951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a56756-8954-4551-b0a1-005407657520 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:33.656569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd313308-93a2-494d-a1ad-e900ed06671b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:33.657217 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd313308-93a2-494d-a1ad-e900ed06671b service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:33.657386 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd313308-93a2-494d-a1ad-e900ed06671b service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:33.657583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd313308-93a2-494d-a1ad-e900ed06671b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:33.659712 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cd313308-93a2-494d-a1ad-e900ed06671b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:33.676843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd313308-93a2-494d-a1ad-e900ed06671b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:33.679182 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd313308-93a2-494d-a1ad-e900ed06671b service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:33.699517 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1058/2128] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:53:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:53:33.894598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c644969-442e-4187-9349-54795fd6612c None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:33.895277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c644969-442e-4187-9349-54795fd6612c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:33.895277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c644969-442e-4187-9349-54795fd6612c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:33.919311 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-4c644969-442e-4187-9349-54795fd6612c None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:53:33.920568 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4c644969-442e-4187-9349-54795fd6612c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:33.929102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1071/2129] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:53:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:53:33.956897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd313308-93a2-494d-a1ad-e900ed06671b service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:33.972071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd313308-93a2-494d-a1ad-e900ed06671b service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:33.977996 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ae59c-98ef-4253-8265-10d8e2b7c3e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:33.978998 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ae59c-98ef-4253-8265-10d8e2b7c3e6 service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:33.979322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ae59c-98ef-4253-8265-10d8e2b7c3e6 service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:33.979631 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ae59c-98ef-4253-8265-10d8e2b7c3e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:33.990492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ae59c-98ef-4253-8265-10d8e2b7c3e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:33.992125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ae59c-98ef-4253-8265-10d8e2b7c3e6 service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:34.001703 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1059/2130] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:53:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:53:43.419063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f93ee-5324-43d0-b00f-d8f6e29c7e29 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:43.419063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f93ee-5324-43d0-b00f-d8f6e29c7e29 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:43.419063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f93ee-5324-43d0-b00f-d8f6e29c7e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:43.442796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-ce4f93ee-5324-43d0-b00f-d8f6e29c7e29 None None] MFA Rules not processed for user `193bf7a17f544770a27c07f6b8cc0988`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:53:43.444133 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ce4f93ee-5324-43d0-b00f-d8f6e29c7e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:43.456849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1072/2131] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:53:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:53:43.997411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=47 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:53:53.436639 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=54 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:53:57.409454 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ae59c-98ef-4253-8265-10d8e2b7c3e6 service swift] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:57.411024 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5d7ae59c-98ef-4253-8265-10d8e2b7c3e6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:57.423176 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ae59c-98ef-4253-8265-10d8e2b7c3e6 service swift] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:57.427638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83122997-ff1d-43d2-9875-737059583cf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:57.428284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83122997-ff1d-43d2-9875-737059583cf6 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:57.429026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83122997-ff1d-43d2-9875-737059583cf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:57.429026 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83122997-ff1d-43d2-9875-737059583cf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:57.431157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-83122997-ff1d-43d2-9875-737059583cf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:57.497216 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83122997-ff1d-43d2-9875-737059583cf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:57.498671 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83122997-ff1d-43d2-9875-737059583cf6 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:57.509194 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1060/2132] 10.222.0.36 () {60 vars in 1396 bytes} [Sat Oct 15 13:53:57 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:53:59.160921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f93ee-5324-43d0-b00f-d8f6e29c7e29 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:53:59.175626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f93ee-5324-43d0-b00f-d8f6e29c7e29 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:53:59.183020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b4612-df26-4341-a3d0-466e64ff4ea7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:53:59.183667 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b4612-df26-4341-a3d0-466e64ff4ea7 service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:53:59.183974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b4612-df26-4341-a3d0-466e64ff4ea7 service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:53:59.184249 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b4612-df26-4341-a3d0-466e64ff4ea7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:53:59.187185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-786b4612-df26-4341-a3d0-466e64ff4ea7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:53:59.274024 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b4612-df26-4341-a3d0-466e64ff4ea7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:53:59.275958 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b4612-df26-4341-a3d0-466e64ff4ea7 service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:53:59.296095 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1073/2133] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:53:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:54:07.504822 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=28 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:54:09.283894 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=28 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:54:14.867462 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eebf3-a1a4-4997-b93a-aea834484e81 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:54:14.868165 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eebf3-a1a4-4997-b93a-aea834484e81 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:54:14.868165 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18eebf3-a1a4-4997-b93a-aea834484e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:54:14.891792 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f18eebf3-a1a4-4997-b93a-aea834484e81 None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:54:14.893085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f18eebf3-a1a4-4997-b93a-aea834484e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:54:14.903663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1061/2134] 10.222.0.36 () {58 vars in 967 bytes} [Sat Oct 15 13:54:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:54:14.918423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b4612-df26-4341-a3d0-466e64ff4ea7 service nova] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:54:14.931415 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b4612-df26-4341-a3d0-466e64ff4ea7 service nova] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:54:14.937251 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa37ad-b306-4482-91df-ee36cf628042 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:54:14.937900 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa37ad-b306-4482-91df-ee36cf628042 service swift] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:54:14.938176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa37ad-b306-4482-91df-ee36cf628042 service swift] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:54:14.938447 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa37ad-b306-4482-91df-ee36cf628042 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:54:14.948200 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa37ad-b306-4482-91df-ee36cf628042 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:54:14.949839 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa37ad-b306-4482-91df-ee36cf628042 service swift] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:54:14.959958 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1074/2135] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:54:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:54:24.884715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:54:24.955780 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=6 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:54:27.800918 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18eebf3-a1a4-4997-b93a-aea834484e81 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:54:27.819811 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18eebf3-a1a4-4997-b93a-aea834484e81 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:54:27.829643 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d8082-7731-4d0e-ba06-2cb420bad0bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:54:27.832306 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d8082-7731-4d0e-ba06-2cb420bad0bc service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:54:27.832986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d8082-7731-4d0e-ba06-2cb420bad0bc service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:54:27.834577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d8082-7731-4d0e-ba06-2cb420bad0bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:54:27.838634 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f00d8082-7731-4d0e-ba06-2cb420bad0bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:54:27.882761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d8082-7731-4d0e-ba06-2cb420bad0bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:54:27.885011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d8082-7731-4d0e-ba06-2cb420bad0bc service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:54:27.903396 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1062/2136] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:54:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:54:32.434942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa37ad-b306-4482-91df-ee36cf628042 service swift] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:54:32.445812 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa37ad-b306-4482-91df-ee36cf628042 service swift] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:54:32.450837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da33b3f-bb9b-4368-877b-5d3d073cb711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:54:32.451554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da33b3f-bb9b-4368-877b-5d3d073cb711 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:54:32.451961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da33b3f-bb9b-4368-877b-5d3d073cb711 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:54:32.452405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da33b3f-bb9b-4368-877b-5d3d073cb711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:54:32.463431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da33b3f-bb9b-4368-877b-5d3d073cb711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:54:32.464924 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da33b3f-bb9b-4368-877b-5d3d073cb711 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:54:32.476517 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1075/2137] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:54:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:54:34.584303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d8082-7731-4d0e-ba06-2cb420bad0bc service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:54:34.585867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f00d8082-7731-4d0e-ba06-2cb420bad0bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:54:34.608541 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d8082-7731-4d0e-ba06-2cb420bad0bc service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:54:34.616547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1a4a0-a5b0-4420-96b5-bd8ae09cf5df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:54:34.617385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1a4a0-a5b0-4420-96b5-bd8ae09cf5df service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:54:34.617796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1a4a0-a5b0-4420-96b5-bd8ae09cf5df service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:54:34.618159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1a4a0-a5b0-4420-96b5-bd8ae09cf5df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:54:34.620591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-95a1a4a0-a5b0-4420-96b5-bd8ae09cf5df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:54:34.659186 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1a4a0-a5b0-4420-96b5-bd8ae09cf5df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:54:34.660986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1a4a0-a5b0-4420-96b5-bd8ae09cf5df service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:54:34.673988 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1063/2138] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:54:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:54:38.793132 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da33b3f-bb9b-4368-877b-5d3d073cb711 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:54:38.810164 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da33b3f-bb9b-4368-877b-5d3d073cb711 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:54:38.817893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b88a6-282e-4b0a-b585-2b9e503875ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:54:38.818736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b88a6-282e-4b0a-b585-2b9e503875ba service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:54:38.819072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b88a6-282e-4b0a-b585-2b9e503875ba service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:54:38.819433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b88a6-282e-4b0a-b585-2b9e503875ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:54:38.822493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2b8b88a6-282e-4b0a-b585-2b9e503875ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:54:38.822973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440e6d5-9ada-4f8c-a24d-f075781abf23 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:54:38.823433 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440e6d5-9ada-4f8c-a24d-f075781abf23 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:54:38.823554 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440e6d5-9ada-4f8c-a24d-f075781abf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:54:38.839391 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b88a6-282e-4b0a-b585-2b9e503875ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:54:38.841897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b88a6-282e-4b0a-b585-2b9e503875ba service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:54:38.855506 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1076/2139] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:54:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:54:38.862348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9440e6d5-9ada-4f8c-a24d-f075781abf23 None None] MFA Rules not processed for user `5150c06020594efe957359f641eba5da`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:54:38.865160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9440e6d5-9ada-4f8c-a24d-f075781abf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:54:38.902446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1064/2140] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:54:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:54:46.225146 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b88a6-282e-4b0a-b585-2b9e503875ba service nova] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:54:46.239086 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b88a6-282e-4b0a-b585-2b9e503875ba service nova] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:54:46.245987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f943097-2564-4779-b59f-fbdd0195cc2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:54:46.246642 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f943097-2564-4779-b59f-fbdd0195cc2e service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:54:46.246920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f943097-2564-4779-b59f-fbdd0195cc2e service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:54:46.247209 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f943097-2564-4779-b59f-fbdd0195cc2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:54:46.249383 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6f943097-2564-4779-b59f-fbdd0195cc2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:54:46.349583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f943097-2564-4779-b59f-fbdd0195cc2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:54:46.352269 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f943097-2564-4779-b59f-fbdd0195cc2e service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:54:46.370577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1077/2141] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:54:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:54:48.852242 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=15 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:54:52.366960 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83794d91-bb0d-4ecf-a23a-f7a28a63df44 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:54:52.367976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83794d91-bb0d-4ecf-a23a-f7a28a63df44 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:54:52.367976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83794d91-bb0d-4ecf-a23a-f7a28a63df44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:54:52.402085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-83794d91-bb0d-4ecf-a23a-f7a28a63df44 None None] MFA Rules not processed for user `707bd577eee14892990367f396270f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:54:52.405991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-83794d91-bb0d-4ecf-a23a-f7a28a63df44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:54:52.415335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1065/2142] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:54:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:54:52.710813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f943097-2564-4779-b59f-fbdd0195cc2e service nova] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:54:52.722990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f943097-2564-4779-b59f-fbdd0195cc2e service nova] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:54:52.728217 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02465611-5486-460f-9435-61db6d934fc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:54:52.729135 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02465611-5486-460f-9435-61db6d934fc9 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:54:52.729796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02465611-5486-460f-9435-61db6d934fc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:54:52.730099 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02465611-5486-460f-9435-61db6d934fc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:54:52.733257 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-02465611-5486-460f-9435-61db6d934fc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:54:52.746690 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02465611-5486-460f-9435-61db6d934fc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:54:52.748768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02465611-5486-460f-9435-61db6d934fc9 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:54:52.761034 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1078/2143] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:54:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:54:53.283778 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83794d91-bb0d-4ecf-a23a-f7a28a63df44 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:54:53.285578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-83794d91-bb0d-4ecf-a23a-f7a28a63df44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:54:53.371220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83794d91-bb0d-4ecf-a23a-f7a28a63df44 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:54:53.376689 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841d996-3f84-4a99-b2f4-5530bbee933a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6720518c03413aa8f85e4d6142fe87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:54:53.377316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841d996-3f84-4a99-b2f4-5530bbee933a service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:54:53.377670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841d996-3f84-4a99-b2f4-5530bbee933a service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:54:53.377944 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841d996-3f84-4a99-b2f4-5530bbee933a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:54:53.379956 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c841d996-3f84-4a99-b2f4-5530bbee933a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:54:53.399915 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841d996-3f84-4a99-b2f4-5530bbee933a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:54:53.402390 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841d996-3f84-4a99-b2f4-5530bbee933a service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:54:53.416088 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1066/2144] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:54:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:55:02.754537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=46 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:55:02.863511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02465611-5486-460f-9435-61db6d934fc9 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:55:02.885300 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02465611-5486-460f-9435-61db6d934fc9 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:55:02.895015 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9647c1c-a04a-4383-bfd7-b46d064c3ebc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:55:02.896111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9647c1c-a04a-4383-bfd7-b46d064c3ebc service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:55:02.896584 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9647c1c-a04a-4383-bfd7-b46d064c3ebc service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:55:02.897068 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9647c1c-a04a-4383-bfd7-b46d064c3ebc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:55:02.900418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e9647c1c-a04a-4383-bfd7-b46d064c3ebc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:55:02.922490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9647c1c-a04a-4383-bfd7-b46d064c3ebc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:55:02.925667 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9647c1c-a04a-4383-bfd7-b46d064c3ebc service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:55:02.944840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1079/2145] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:55:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:55:03.410615 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=35 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:55:12.936321 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=6 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:56:00.327736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b5278-148c-4bef-90d9-935ae8be9850 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:00.328492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b5278-148c-4bef-90d9-935ae8be9850 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:00.328492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b5278-148c-4bef-90d9-935ae8be9850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:00.349312 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8b0b5278-148c-4bef-90d9-935ae8be9850 None None] MFA Rules not processed for user `0b01e277944d4edcb8625ebf4544e340`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:56:00.350523 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8b0b5278-148c-4bef-90d9-935ae8be9850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:00.358307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1067/2146] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:56:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:56:00.684161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9647c1c-a04a-4383-bfd7-b46d064c3ebc service nova] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:00.695209 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9647c1c-a04a-4383-bfd7-b46d064c3ebc service nova] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:00.699957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc26ba-9215-42e2-a8e5-f8e7ba507911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:00.700615 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc26ba-9215-42e2-a8e5-f8e7ba507911 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:00.700890 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc26ba-9215-42e2-a8e5-f8e7ba507911 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:00.701152 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc26ba-9215-42e2-a8e5-f8e7ba507911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:00.703394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-43fc26ba-9215-42e2-a8e5-f8e7ba507911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:00.714266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc26ba-9215-42e2-a8e5-f8e7ba507911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:00.715796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc26ba-9215-42e2-a8e5-f8e7ba507911 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:00.723709 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1080/2147] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:56:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:56:02.425490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b5278-148c-4bef-90d9-935ae8be9850 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:02.426890 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8b0b5278-148c-4bef-90d9-935ae8be9850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:02.451489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b5278-148c-4bef-90d9-935ae8be9850 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:02.458711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870948e-680a-4113-a2e4-62793a6611e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6720518c03413aa8f85e4d6142fe87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:02.459513 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870948e-680a-4113-a2e4-62793a6611e0 service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:02.459796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870948e-680a-4113-a2e4-62793a6611e0 service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:02.460183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870948e-680a-4113-a2e4-62793a6611e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:02.462286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2870948e-680a-4113-a2e4-62793a6611e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:02.476913 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870948e-680a-4113-a2e4-62793a6611e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:02.481229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870948e-680a-4113-a2e4-62793a6611e0 service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:02.489832 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc26ba-9215-42e2-a8e5-f8e7ba507911 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:02.501096 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1068/2148] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:56:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:56:02.508249 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc26ba-9215-42e2-a8e5-f8e7ba507911 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:02.514060 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c79c4-147e-414c-8988-90ce6c7c6219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:02.514991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c79c4-147e-414c-8988-90ce6c7c6219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:02.515260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c79c4-147e-414c-8988-90ce6c7c6219 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:02.515718 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c79c4-147e-414c-8988-90ce6c7c6219 admin admin] PATH_INFO: `/v3/users/707bd577eee14892990367f396270f7b` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:02.517033 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c79c4-147e-414c-8988-90ce6c7c6219 admin admin] RBAC: Authorizing `identity:delete_user(user_id=707bd577eee14892990367f396270f7b)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:02.519184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c79c4-147e-414c-8988-90ce6c7c6219 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:02.583763 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-680c79c4-147e-414c-8988-90ce6c7c6219 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '707bd577eee14892990367f396270f7b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:02.593426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-680c79c4-147e-414c-8988-90ce6c7c6219 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '707bd577eee14892990367f396270f7b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:02.593426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-680c79c4-147e-414c-8988-90ce6c7c6219 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '707bd577eee14892990367f396270f7b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:02.594230 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-680c79c4-147e-414c-8988-90ce6c7c6219 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '707bd577eee14892990367f396270f7b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:02.604249 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-680c79c4-147e-414c-8988-90ce6c7c6219 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '707bd577eee14892990367f396270f7b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:02.631139 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1081/2149] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:56:02 2022] DELETE /identity/v3/users/707bd577eee14892990367f396270f7b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:56:02.833874 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870948e-680a-4113-a2e4-62793a6611e0 service placement] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:02.846564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870948e-680a-4113-a2e4-62793a6611e0 service placement] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:02.912594 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414cc72-7498-4073-930f-7544359b6e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:02.913244 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414cc72-7498-4073-930f-7544359b6e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:02.913568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414cc72-7498-4073-930f-7544359b6e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:02.913821 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414cc72-7498-4073-930f-7544359b6e27 admin admin] PATH_INFO: `/v3/projects/190914cc865b4003ab4b735d937e130d` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:02.914934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414cc72-7498-4073-930f-7544359b6e27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=190914cc865b4003ab4b735d937e130d)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:02.916969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414cc72-7498-4073-930f-7544359b6e27 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:02.981622 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d414cc72-7498-4073-930f-7544359b6e27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '190914cc865b4003ab4b735d937e130d', 'request_id': 'req-d414cc72-7498-4073-930f-7544359b6e27'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:02.990835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d414cc72-7498-4073-930f-7544359b6e27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '190914cc865b4003ab4b735d937e130d', 'request_id': 'req-d414cc72-7498-4073-930f-7544359b6e27'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:03.003160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d414cc72-7498-4073-930f-7544359b6e27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '190914cc865b4003ab4b735d937e130d', 'request_id': 'req-d414cc72-7498-4073-930f-7544359b6e27'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:03.018341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d414cc72-7498-4073-930f-7544359b6e27 admin admin] The token cache is being invalidate because project 190914cc865b4003ab4b735d937e130d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:56:03.018794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d414cc72-7498-4073-930f-7544359b6e27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:03.019797 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d414cc72-7498-4073-930f-7544359b6e27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:03.022774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1069/2150] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:56:02 2022] DELETE /identity/v3/projects/190914cc865b4003ab4b735d937e130d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:56:12.627295 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:56:13.014516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:56:34.189526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820bf64-9023-4c2d-8ef8-363e58033cd8 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:34.190523 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820bf64-9023-4c2d-8ef8-363e58033cd8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:34.190523 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820bf64-9023-4c2d-8ef8-363e58033cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:34.220298 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-a820bf64-9023-4c2d-8ef8-363e58033cd8 None None] MFA Rules not processed for user `583cce1547144e11978f20a3193ceba4`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:56:34.222428 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-a820bf64-9023-4c2d-8ef8-363e58033cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:34.297500 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1082/2151] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:56:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:56:37.083297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1c622-cfae-4780-aed4-61b5ae631e86 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:37.083904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1c622-cfae-4780-aed4-61b5ae631e86 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:37.083904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1c622-cfae-4780-aed4-61b5ae631e86 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:37.085475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1070/2152] 10.222.0.36 () {54 vars in 921 bytes} [Sat Oct 15 13:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:56:37.092281 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58748485-736e-4e21-a0ec-ab2af7528ca2 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:37.092838 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58748485-736e-4e21-a0ec-ab2af7528ca2 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:37.093070 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58748485-736e-4e21-a0ec-ab2af7528ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:37.135448 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-58748485-736e-4e21-a0ec-ab2af7528ca2 None None] MFA Rules not processed for user `90985ec8c43a4d7895942731dc1d002c`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:56:37.136754 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-58748485-736e-4e21-a0ec-ab2af7528ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:37.216997 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1083/2153] 10.222.0.36 () {58 vars in 1018 bytes} [Sat Oct 15 13:56:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:56:37.234740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1c622-cfae-4780-aed4-61b5ae631e86 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:37.244469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-23d1c622-cfae-4780-aed4-61b5ae631e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:37.266816 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1c622-cfae-4780-aed4-61b5ae631e86 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:37.271673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2240e7e-a8cd-4fa0-a1be-cc1e48d4b3f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:37.272376 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2240e7e-a8cd-4fa0-a1be-cc1e48d4b3f9 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:37.272848 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2240e7e-a8cd-4fa0-a1be-cc1e48d4b3f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:37.273322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2240e7e-a8cd-4fa0-a1be-cc1e48d4b3f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:37.284312 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2240e7e-a8cd-4fa0-a1be-cc1e48d4b3f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:37.287287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2240e7e-a8cd-4fa0-a1be-cc1e48d4b3f9 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:37.303639 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1071/2154] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:56:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:56:37.410540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bae2d2-3d9a-49df-b106-958c493e0e9e None None] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:37.411256 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bae2d2-3d9a-49df-b106-958c493e0e9e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:37.411756 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bae2d2-3d9a-49df-b106-958c493e0e9e None None] PATH_INFO: `/` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:37.416183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1084/2155] 10.222.0.36 () {54 vars in 921 bytes} [Sat Oct 15 13:56:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 15 13:56:37.421583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac0fd8-6a16-42ee-8eb5-f71b91ed3f01 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:37.421813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac0fd8-6a16-42ee-8eb5-f71b91ed3f01 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:37.422074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac0fd8-6a16-42ee-8eb5-f71b91ed3f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:37.482991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-3bac0fd8-6a16-42ee-8eb5-f71b91ed3f01 None None] MFA Rules not processed for user `2e13692c1e83413a909d99bb7db2e4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:56:37.484774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3bac0fd8-6a16-42ee-8eb5-f71b91ed3f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:37.655256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1072/2156] 10.222.0.36 () {58 vars in 1018 bytes} [Sat Oct 15 13:56:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:56:37.674132 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bae2d2-3d9a-49df-b106-958c493e0e9e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:37.694724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bae2d2-3d9a-49df-b106-958c493e0e9e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:37.703012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8a084-5f61-4e59-8bb6-51d04df99bcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:37.704093 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8a084-5f61-4e59-8bb6-51d04df99bcc service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:37.704577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8a084-5f61-4e59-8bb6-51d04df99bcc service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:37.705074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8a084-5f61-4e59-8bb6-51d04df99bcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:37.727407 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8a084-5f61-4e59-8bb6-51d04df99bcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:37.730865 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8a084-5f61-4e59-8bb6-51d04df99bcc service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:37.748502 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1085/2157] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:56:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:56:38.974402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac0fd8-6a16-42ee-8eb5-f71b91ed3f01 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:38.987518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac0fd8-6a16-42ee-8eb5-f71b91ed3f01 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:38.993587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7abc9-52a6-45b9-8770-c849bf9e297e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:38.994223 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7abc9-52a6-45b9-8770-c849bf9e297e service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:38.994497 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7abc9-52a6-45b9-8770-c849bf9e297e service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:38.994769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7abc9-52a6-45b9-8770-c849bf9e297e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:38.996672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-abb7abc9-52a6-45b9-8770-c849bf9e297e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:39.130017 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7abc9-52a6-45b9-8770-c849bf9e297e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:39.131497 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7abc9-52a6-45b9-8770-c849bf9e297e service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:39.147954 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:56:39.157891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1073/2158] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:56:39.669774 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8a084-5f61-4e59-8bb6-51d04df99bcc service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:39.672204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c3e8a084-5f61-4e59-8bb6-51d04df99bcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:39.686264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8a084-5f61-4e59-8bb6-51d04df99bcc service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:39.691301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ffeca-8e63-43ff-bbe4-0274c06c35f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:39.691977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffeca-8e63-43ff-bbe4-0274c06c35f7 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:39.692257 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffeca-8e63-43ff-bbe4-0274c06c35f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:39.692528 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffeca-8e63-43ff-bbe4-0274c06c35f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:39.695161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f91ffeca-8e63-43ff-bbe4-0274c06c35f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:39.779935 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ffeca-8e63-43ff-bbe4-0274c06c35f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:39.782005 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ffeca-8e63-43ff-bbe4-0274c06c35f7 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:39.794583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1086/2159] 10.222.0.36 () {60 vars in 1396 bytes} [Sat Oct 15 13:56:39 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 13:56:44.394690 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7abc9-52a6-45b9-8770-c849bf9e297e service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:44.398191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-abb7abc9-52a6-45b9-8770-c849bf9e297e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:44.422163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7abc9-52a6-45b9-8770-c849bf9e297e service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:44.428041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53131cb-aa54-48c1-8826-36e902ccc78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:44.428814 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53131cb-aa54-48c1-8826-36e902ccc78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:44.429143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53131cb-aa54-48c1-8826-36e902ccc78b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:44.429557 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53131cb-aa54-48c1-8826-36e902ccc78b admin admin] PATH_INFO: `/v3/users/5150c06020594efe957359f641eba5da` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:44.431377 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53131cb-aa54-48c1-8826-36e902ccc78b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5150c06020594efe957359f641eba5da)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:44.434538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53131cb-aa54-48c1-8826-36e902ccc78b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:44.508545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a53131cb-aa54-48c1-8826-36e902ccc78b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5150c06020594efe957359f641eba5da'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:44.509431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a53131cb-aa54-48c1-8826-36e902ccc78b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5150c06020594efe957359f641eba5da'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:44.509852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a53131cb-aa54-48c1-8826-36e902ccc78b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5150c06020594efe957359f641eba5da'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:44.525814 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a53131cb-aa54-48c1-8826-36e902ccc78b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5150c06020594efe957359f641eba5da'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:44.538818 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a53131cb-aa54-48c1-8826-36e902ccc78b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5150c06020594efe957359f641eba5da'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:44.555742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1074/2160] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:56:44 2022] DELETE /identity/v3/users/5150c06020594efe957359f641eba5da => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:56:44.760877 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ffeca-8e63-43ff-bbe4-0274c06c35f7 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:44.766968 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:56:44.776588 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ffeca-8e63-43ff-bbe4-0274c06c35f7 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:44.856649 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab9d07-75ef-4f1d-8836-11eb9be79f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:44.857336 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab9d07-75ef-4f1d-8836-11eb9be79f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:44.857648 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab9d07-75ef-4f1d-8836-11eb9be79f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:44.857923 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab9d07-75ef-4f1d-8836-11eb9be79f16 admin admin] PATH_INFO: `/v3/projects/09a0813bae074cd2aa552be82abe8493` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:44.859120 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab9d07-75ef-4f1d-8836-11eb9be79f16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09a0813bae074cd2aa552be82abe8493)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:44.861208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab9d07-75ef-4f1d-8836-11eb9be79f16 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:44.935221 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9fab9d07-75ef-4f1d-8836-11eb9be79f16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09a0813bae074cd2aa552be82abe8493', 'request_id': 'req-9fab9d07-75ef-4f1d-8836-11eb9be79f16'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:44.948537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9fab9d07-75ef-4f1d-8836-11eb9be79f16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09a0813bae074cd2aa552be82abe8493', 'request_id': 'req-9fab9d07-75ef-4f1d-8836-11eb9be79f16'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:44.956972 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9fab9d07-75ef-4f1d-8836-11eb9be79f16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09a0813bae074cd2aa552be82abe8493', 'request_id': 'req-9fab9d07-75ef-4f1d-8836-11eb9be79f16'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:44.969167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9fab9d07-75ef-4f1d-8836-11eb9be79f16 admin admin] The token cache is being invalidate because project 09a0813bae074cd2aa552be82abe8493 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:56:44.969472 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9fab9d07-75ef-4f1d-8836-11eb9be79f16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:44.970142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9fab9d07-75ef-4f1d-8836-11eb9be79f16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:56:44.971687 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1087/2161] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:56:44 2022] DELETE /identity/v3/projects/09a0813bae074cd2aa552be82abe8493 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:56:44.977859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d0fe5-4118-4a71-aa3c-2ceda84b782e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:44.978226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d0fe5-4118-4a71-aa3c-2ceda84b782e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:44.978555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d0fe5-4118-4a71-aa3c-2ceda84b782e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:45.032728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-252d0fe5-4118-4a71-aa3c-2ceda84b782e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:56:45.034059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-252d0fe5-4118-4a71-aa3c-2ceda84b782e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:45.110550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1075/2162] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:56:45.113700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ce9ac-d116-4381-88f7-b940f22bd72a None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:45.114039 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ce9ac-d116-4381-88f7-b940f22bd72a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:45.114665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ce9ac-d116-4381-88f7-b940f22bd72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:45.149735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-9a3ce9ac-d116-4381-88f7-b940f22bd72a None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:56:45.152078 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9a3ce9ac-d116-4381-88f7-b940f22bd72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:45.166668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1088/2163] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:56:45.173673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d0fe5-4118-4a71-aa3c-2ceda84b782e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:45.191781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d0fe5-4118-4a71-aa3c-2ceda84b782e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:45.197139 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc554af8-3a21-46f5-921f-2502f5f0a131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:45.197776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc554af8-3a21-46f5-921f-2502f5f0a131 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:45.198056 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc554af8-3a21-46f5-921f-2502f5f0a131 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:45.198324 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc554af8-3a21-46f5-921f-2502f5f0a131 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:45.199159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc554af8-3a21-46f5-921f-2502f5f0a131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:56:45.199460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc554af8-3a21-46f5-921f-2502f5f0a131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:45.201168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc554af8-3a21-46f5-921f-2502f5f0a131 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:45.217059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1076/2164] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:56:45.222885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ce9ac-d116-4381-88f7-b940f22bd72a None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:45.241089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ce9ac-d116-4381-88f7-b940f22bd72a None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:45.249197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787322c-54c5-4bba-b780-2f26881d32e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:45.250323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787322c-54c5-4bba-b780-2f26881d32e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:45.250870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787322c-54c5-4bba-b780-2f26881d32e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:45.251418 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787322c-54c5-4bba-b780-2f26881d32e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:45.252696 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787322c-54c5-4bba-b780-2f26881d32e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:45.256664 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787322c-54c5-4bba-b780-2f26881d32e5 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:45.303917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1089/2165] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:56:45 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:56:45.306199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc554af8-3a21-46f5-921f-2502f5f0a131 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:45.319574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc554af8-3a21-46f5-921f-2502f5f0a131 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:45.374613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:56:45.411486 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac9fa0-3e7b-4737-97fb-78dbbe35423a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:45.412168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac9fa0-3e7b-4737-97fb-78dbbe35423a admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:45.412404 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac9fa0-3e7b-4737-97fb-78dbbe35423a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:45.412669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac9fa0-3e7b-4737-97fb-78dbbe35423a admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:45.413316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac9fa0-3e7b-4737-97fb-78dbbe35423a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:45.415335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac9fa0-3e7b-4737-97fb-78dbbe35423a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:45.440181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1077/2166] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:56:45 2022] POST /identity/v3/users => generated 291 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 13:56:45.447288 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787322c-54c5-4bba-b780-2f26881d32e5 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:45.459747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787322c-54c5-4bba-b780-2f26881d32e5 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:45.467585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffd054-df49-4b32-a306-d3e64f9aec97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:45.468290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffd054-df49-4b32-a306-d3e64f9aec97 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:45.468568 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffd054-df49-4b32-a306-d3e64f9aec97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:45.468837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffd054-df49-4b32-a306-d3e64f9aec97 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:45.469505 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ffd054-df49-4b32-a306-d3e64f9aec97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:45.471189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ffd054-df49-4b32-a306-d3e64f9aec97 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:45.484342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1090/2167] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:56:45.489106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac9fa0-3e7b-4737-97fb-78dbbe35423a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:45.501360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac9fa0-3e7b-4737-97fb-78dbbe35423a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:45.507597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2f554-e50c-4312-a63c-6ccb6ebe49e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:45.508452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f554-e50c-4312-a63c-6ccb6ebe49e8 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:45.508887 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f554-e50c-4312-a63c-6ccb6ebe49e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:45.509355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f554-e50c-4312-a63c-6ccb6ebe49e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:45.510043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2f554-e50c-4312-a63c-6ccb6ebe49e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:45.511780 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2f554-e50c-4312-a63c-6ccb6ebe49e8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:45.530199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1078/2168] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:56:45.536835 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffd054-df49-4b32-a306-d3e64f9aec97 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:45.549019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffd054-df49-4b32-a306-d3e64f9aec97 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:45.554685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c33a79-7da0-4218-b61c-c78649a4cee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:45.555465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c33a79-7da0-4218-b61c-c78649a4cee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:45.555730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c33a79-7da0-4218-b61c-c78649a4cee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:45.556109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c33a79-7da0-4218-b61c-c78649a4cee1 admin admin] PATH_INFO: `/v3/projects/1976b27e84a4484ba9d46a4244b08699/users/a10fc1a48c9c41eaa1bf3ff2e4109657/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:45.574899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c33a79-7da0-4218-b61c-c78649a4cee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1976b27e84a4484ba9d46a4244b08699, user_id=a10fc1a48c9c41eaa1bf3ff2e4109657, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:45.578139 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c33a79-7da0-4218-b61c-c78649a4cee1 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:45.595065 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1091/2169] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:56:45 2022] PUT /identity/v3/projects/1976b27e84a4484ba9d46a4244b08699/users/a10fc1a48c9c41eaa1bf3ff2e4109657/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:56:45.601246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec7be0-75f8-4aa9-8407-a0043a4ec63e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:45.601702 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec7be0-75f8-4aa9-8407-a0043a4ec63e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:45.602372 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec7be0-75f8-4aa9-8407-a0043a4ec63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:45.624933 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-79ec7be0-75f8-4aa9-8407-a0043a4ec63e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:56:45.626328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-79ec7be0-75f8-4aa9-8407-a0043a4ec63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:45.707932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1079/2170] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:56:45.721013 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c33a79-7da0-4218-b61c-c78649a4cee1 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:45.723913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-54c33a79-7da0-4218-b61c-c78649a4cee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:45.797370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:56:45.821722 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c33a79-7da0-4218-b61c-c78649a4cee1 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:45.825536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed541da5-fd7e-417b-8052-3afd00433c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:45.826165 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed541da5-fd7e-417b-8052-3afd00433c76 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:45.827073 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed541da5-fd7e-417b-8052-3afd00433c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:45.827073 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed541da5-fd7e-417b-8052-3afd00433c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:45.840534 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed541da5-fd7e-417b-8052-3afd00433c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:45.842038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed541da5-fd7e-417b-8052-3afd00433c76 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:45.859043 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1092/2171] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:56:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:56:46.844895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec7be0-75f8-4aa9-8407-a0043a4ec63e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:46.846602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-79ec7be0-75f8-4aa9-8407-a0043a4ec63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:47.012812 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec7be0-75f8-4aa9-8407-a0043a4ec63e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:47.022774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a55b7d-d7e1-4b40-a87e-b15c7d961209 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6720518c03413aa8f85e4d6142fe87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:47.023675 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a55b7d-d7e1-4b40-a87e-b15c7d961209 service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:47.024002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a55b7d-d7e1-4b40-a87e-b15c7d961209 service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:47.024294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a55b7d-d7e1-4b40-a87e-b15c7d961209 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:47.028790 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-88a55b7d-d7e1-4b40-a87e-b15c7d961209 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:47.041099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a55b7d-d7e1-4b40-a87e-b15c7d961209 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:47.042946 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a55b7d-d7e1-4b40-a87e-b15c7d961209 service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:47.056063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1080/2172] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:56:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:56:51.310561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e6c85-74c6-4f1d-a539-6a0e4673f690 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:51.311984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e6c85-74c6-4f1d-a539-6a0e4673f690 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:51.312690 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e6c85-74c6-4f1d-a539-6a0e4673f690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:51.368429 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-949e6c85-74c6-4f1d-a539-6a0e4673f690 None None] MFA Rules not processed for user `a10fc1a48c9c41eaa1bf3ff2e4109657`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:56:51.370991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-949e6c85-74c6-4f1d-a539-6a0e4673f690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:51.552975 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1093/2173] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:56:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:56:51.576187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a55b7d-d7e1-4b40-a87e-b15c7d961209 service placement] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:56:51.578700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-88a55b7d-d7e1-4b40-a87e-b15c7d961209 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:51.642271 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:56:51.735036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a55b7d-d7e1-4b40-a87e-b15c7d961209 service placement] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:56:51.744638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95114e11-e4c6-48ed-a5f2-294f66a7f443 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:56:51.746506 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95114e11-e4c6-48ed-a5f2-294f66a7f443 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:51.746506 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95114e11-e4c6-48ed-a5f2-294f66a7f443 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:51.746911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95114e11-e4c6-48ed-a5f2-294f66a7f443 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:51.764551 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95114e11-e4c6-48ed-a5f2-294f66a7f443 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:56:51.767996 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95114e11-e4c6-48ed-a5f2-294f66a7f443 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:56:51.790583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1081/2174] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:56:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:56:53.917814 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabf24f-8ec6-4b92-bfcd-4e1956f4c59a None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:56:53.918771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabf24f-8ec6-4b92-bfcd-4e1956f4c59a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:56:53.918771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabf24f-8ec6-4b92-bfcd-4e1956f4c59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:56:53.951615 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-2dabf24f-8ec6-4b92-bfcd-4e1956f4c59a None None] MFA Rules not processed for user `a10fc1a48c9c41eaa1bf3ff2e4109657`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:56:53.953173 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2dabf24f-8ec6-4b92-bfcd-4e1956f4c59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:56:53.967691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1094/2175] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:56:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:57:01.780274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=23 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:57:03.946261 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=48 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:57:16.267799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21881a-5d76-4264-b41b-08e8df743e02 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:57:16.268284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21881a-5d76-4264-b41b-08e8df743e02 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:57:16.268374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21881a-5d76-4264-b41b-08e8df743e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:57:16.301101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9c21881a-5d76-4264-b41b-08e8df743e02 None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:57:16.302328 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9c21881a-5d76-4264-b41b-08e8df743e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:57:16.364476 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1082/2176] 10.222.0.36 () {58 vars in 967 bytes} [Sat Oct 15 13:57:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:57:16.418360 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabf24f-8ec6-4b92-bfcd-4e1956f4c59a None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:57:16.419913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2dabf24f-8ec6-4b92-bfcd-4e1956f4c59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:57:16.498261 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabf24f-8ec6-4b92-bfcd-4e1956f4c59a None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:57:16.503198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e5e9e-fade-43d3-ba86-9d06c55c1601 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:57:16.503815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e5e9e-fade-43d3-ba86-9d06c55c1601 service swift] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:57:16.504185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e5e9e-fade-43d3-ba86-9d06c55c1601 service swift] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:57:16.504459 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e5e9e-fade-43d3-ba86-9d06c55c1601 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:57:16.513035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e5e9e-fade-43d3-ba86-9d06c55c1601 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:57:16.514626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e5e9e-fade-43d3-ba86-9d06c55c1601 service swift] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:57:16.524320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1095/2177] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:57:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 15 13:57:16.914183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21881a-5d76-4264-b41b-08e8df743e02 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:57:16.934402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21881a-5d76-4264-b41b-08e8df743e02 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:57:16.938314 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1352b47-2035-4735-a381-e127d868772f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:57:16.938935 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1352b47-2035-4735-a381-e127d868772f service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:57:16.939206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1352b47-2035-4735-a381-e127d868772f service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:57:16.939473 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1352b47-2035-4735-a381-e127d868772f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:57:16.941336 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b1352b47-2035-4735-a381-e127d868772f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:57:16.953921 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1352b47-2035-4735-a381-e127d868772f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:57:16.955436 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1352b47-2035-4735-a381-e127d868772f service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:57:16.965715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1083/2178] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:57:23.865223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e5e9e-fade-43d3-ba86-9d06c55c1601 service swift] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:57:23.882619 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e5e9e-fade-43d3-ba86-9d06c55c1601 service swift] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:57:23.888007 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161e9b2-a67d-4593-9a70-623b9fc4844f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6720518c03413aa8f85e4d6142fe87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:57:23.888628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161e9b2-a67d-4593-9a70-623b9fc4844f service placement] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:57:23.888931 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161e9b2-a67d-4593-9a70-623b9fc4844f service placement] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:57:23.889169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161e9b2-a67d-4593-9a70-623b9fc4844f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:57:23.891401 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b161e9b2-a67d-4593-9a70-623b9fc4844f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:57:23.903765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161e9b2-a67d-4593-9a70-623b9fc4844f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:57:23.905380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161e9b2-a67d-4593-9a70-623b9fc4844f service placement] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:57:23.917477 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1096/2179] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:57:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:57:23.925261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1352b47-2035-4735-a381-e127d868772f service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:57:23.927248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b1352b47-2035-4735-a381-e127d868772f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:57:23.942565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1352b47-2035-4735-a381-e127d868772f service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:57:23.949952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d21b43-fd75-41b0-8960-060714b1910c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:57:23.950612 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d21b43-fd75-41b0-8960-060714b1910c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:57:23.950888 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d21b43-fd75-41b0-8960-060714b1910c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:57:23.951153 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d21b43-fd75-41b0-8960-060714b1910c admin admin] PATH_INFO: `/v3/users/dc83c1c8fce04c57a685de883344b7e6` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:57:23.952434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d21b43-fd75-41b0-8960-060714b1910c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc83c1c8fce04c57a685de883344b7e6)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:57:23.954562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d21b43-fd75-41b0-8960-060714b1910c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:57:24.019432 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-57d21b43-fd75-41b0-8960-060714b1910c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc83c1c8fce04c57a685de883344b7e6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.020195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-57d21b43-fd75-41b0-8960-060714b1910c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc83c1c8fce04c57a685de883344b7e6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.020814 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-57d21b43-fd75-41b0-8960-060714b1910c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc83c1c8fce04c57a685de883344b7e6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.033748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-57d21b43-fd75-41b0-8960-060714b1910c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc83c1c8fce04c57a685de883344b7e6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.043528 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-57d21b43-fd75-41b0-8960-060714b1910c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc83c1c8fce04c57a685de883344b7e6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.057233 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1084/2180] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:57:23 2022] DELETE /identity/v3/users/dc83c1c8fce04c57a685de883344b7e6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:57:24.253306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161e9b2-a67d-4593-9a70-623b9fc4844f service placement] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:57:24.264821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161e9b2-a67d-4593-9a70-623b9fc4844f service placement] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:57:24.323955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:57:24.324599 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:57:24.324874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:57:24.325142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] PATH_INFO: `/v3/projects/e752c3aef55649ea98ec1a2f3607c27f` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:57:24.326518 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e752c3aef55649ea98ec1a2f3607c27f)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:57:24.328453 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:57:24.381711 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e752c3aef55649ea98ec1a2f3607c27f', 'request_id': 'req-9359a830-3a96-4c63-8204-fbd4b5f41ae5'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.392214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e752c3aef55649ea98ec1a2f3607c27f', 'request_id': 'req-9359a830-3a96-4c63-8204-fbd4b5f41ae5'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.399520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e752c3aef55649ea98ec1a2f3607c27f', 'request_id': 'req-9359a830-3a96-4c63-8204-fbd4b5f41ae5'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.412098 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] The token cache is being invalidate because project e752c3aef55649ea98ec1a2f3607c27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:57:24.412383 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.413069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.414623 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1097/2181] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:57:24 2022] DELETE /identity/v3/projects/e752c3aef55649ea98ec1a2f3607c27f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:57:24.418866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d21b43-fd75-41b0-8960-060714b1910c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:57:24.420802 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-57d21b43-fd75-41b0-8960-060714b1910c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:57:24.434281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d21b43-fd75-41b0-8960-060714b1910c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:57:24.487903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:57:24.504024 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adac66-f1f3-46bf-b5fc-fe1a778dcd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:57:24.504710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adac66-f1f3-46bf-b5fc-fe1a778dcd44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:57:24.504985 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adac66-f1f3-46bf-b5fc-fe1a778dcd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:57:24.505272 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adac66-f1f3-46bf-b5fc-fe1a778dcd44 admin admin] PATH_INFO: `/v3/users/4413b1d895c9438c8f8fe01408e53e8b` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:57:24.506456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41adac66-f1f3-46bf-b5fc-fe1a778dcd44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4413b1d895c9438c8f8fe01408e53e8b)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:57:24.508456 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41adac66-f1f3-46bf-b5fc-fe1a778dcd44 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:57:24.565767 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-41adac66-f1f3-46bf-b5fc-fe1a778dcd44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4413b1d895c9438c8f8fe01408e53e8b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.566428 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-41adac66-f1f3-46bf-b5fc-fe1a778dcd44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4413b1d895c9438c8f8fe01408e53e8b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.567183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-41adac66-f1f3-46bf-b5fc-fe1a778dcd44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4413b1d895c9438c8f8fe01408e53e8b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.581407 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-41adac66-f1f3-46bf-b5fc-fe1a778dcd44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4413b1d895c9438c8f8fe01408e53e8b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.592450 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-41adac66-f1f3-46bf-b5fc-fe1a778dcd44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4413b1d895c9438c8f8fe01408e53e8b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.609055 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1085/2182] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:57:24 2022] DELETE /identity/v3/users/4413b1d895c9438c8f8fe01408e53e8b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:57:24.804828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:57:24.812691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:57:24.826979 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359a830-3a96-4c63-8204-fbd4b5f41ae5 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:57:24.831381 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:57:24.897784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f55f90-a329-4b52-9739-163576f7af66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:57:24.898534 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f55f90-a329-4b52-9739-163576f7af66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:57:24.898884 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f55f90-a329-4b52-9739-163576f7af66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:57:24.899182 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f55f90-a329-4b52-9739-163576f7af66 admin admin] PATH_INFO: `/v3/projects/c15ea1a4d30742a2983b585cf7aa4eff` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:57:24.901327 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f55f90-a329-4b52-9739-163576f7af66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15ea1a4d30742a2983b585cf7aa4eff)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:57:24.905020 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f55f90-a329-4b52-9739-163576f7af66 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:57:24.956158 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c8f55f90-a329-4b52-9739-163576f7af66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15ea1a4d30742a2983b585cf7aa4eff', 'request_id': 'req-c8f55f90-a329-4b52-9739-163576f7af66'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.965706 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c8f55f90-a329-4b52-9739-163576f7af66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15ea1a4d30742a2983b585cf7aa4eff', 'request_id': 'req-c8f55f90-a329-4b52-9739-163576f7af66'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.973132 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c8f55f90-a329-4b52-9739-163576f7af66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15ea1a4d30742a2983b585cf7aa4eff', 'request_id': 'req-c8f55f90-a329-4b52-9739-163576f7af66'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.989591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c8f55f90-a329-4b52-9739-163576f7af66 admin admin] The token cache is being invalidate because project c15ea1a4d30742a2983b585cf7aa4eff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:57:24.989591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c8f55f90-a329-4b52-9739-163576f7af66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.989591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c8f55f90-a329-4b52-9739-163576f7af66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:57:24.989591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1098/2183] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:57:24 2022] DELETE /identity/v3/projects/c15ea1a4d30742a2983b585cf7aa4eff => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:57:34.604740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=32,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:57:34.980725 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:57:35.713007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46965ecf-e975-49ad-8000-8f8159303fb6 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:57:35.713790 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46965ecf-e975-49ad-8000-8f8159303fb6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:57:35.713790 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46965ecf-e975-49ad-8000-8f8159303fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:57:35.738696 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-46965ecf-e975-49ad-8000-8f8159303fb6 None None] MFA Rules not processed for user `193bf7a17f544770a27c07f6b8cc0988`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:57:35.740183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-46965ecf-e975-49ad-8000-8f8159303fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:57:35.822597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1086/2184] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:57:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:57:45.813925 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=25 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:58:03.241253 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2b0c4-582d-4ae8-95dc-2dcad75ff65a None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:58:03.241746 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2b0c4-582d-4ae8-95dc-2dcad75ff65a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:58:03.241849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2b0c4-582d-4ae8-95dc-2dcad75ff65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:58:03.264406 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-4ed2b0c4-582d-4ae8-95dc-2dcad75ff65a None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:58:03.265639 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4ed2b0c4-582d-4ae8-95dc-2dcad75ff65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:58:03.340613 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1099/2185] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:58:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:58:03.353495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46965ecf-e975-49ad-8000-8f8159303fb6 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:58:03.355262 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-46965ecf-e975-49ad-8000-8f8159303fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:58:03.431553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46965ecf-e975-49ad-8000-8f8159303fb6 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:58:03.437023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcd6a1-6442-427f-a688-95d7b750e3fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:58:03.437642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcd6a1-6442-427f-a688-95d7b750e3fc service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:58:03.437916 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcd6a1-6442-427f-a688-95d7b750e3fc service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:58:03.438181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcd6a1-6442-427f-a688-95d7b750e3fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:58:03.447897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcd6a1-6442-427f-a688-95d7b750e3fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:58:03.449871 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcd6a1-6442-427f-a688-95d7b750e3fc service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:58:03.458754 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1087/2186] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:58:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:58:13.324982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=25 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:58:13.455089 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=32 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:58:20.228908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf7e0f-43d9-4178-807d-c49df48d4f75 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:58:20.229461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf7e0f-43d9-4178-807d-c49df48d4f75 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:58:20.229461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf7e0f-43d9-4178-807d-c49df48d4f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:58:20.253817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-0edf7e0f-43d9-4178-807d-c49df48d4f75 None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:58:20.254947 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0edf7e0f-43d9-4178-807d-c49df48d4f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:58:20.263616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1100/2187] 10.222.0.36 () {58 vars in 1009 bytes} [Sat Oct 15 13:58:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:58:20.289956 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcd6a1-6442-427f-a688-95d7b750e3fc service swift] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:58:20.301168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcd6a1-6442-427f-a688-95d7b750e3fc service swift] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:58:20.306827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094df1c-34fb-42e6-9811-8e963a39a245 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:58:20.307440 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094df1c-34fb-42e6-9811-8e963a39a245 service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:58:20.307702 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094df1c-34fb-42e6-9811-8e963a39a245 service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:58:20.308002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094df1c-34fb-42e6-9811-8e963a39a245 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:58:20.317551 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094df1c-34fb-42e6-9811-8e963a39a245 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:58:20.319174 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094df1c-34fb-42e6-9811-8e963a39a245 service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:58:20.328933 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1088/2188] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:58:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:58:30.248454 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:58:30.324442 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=6 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:58:32.787782 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf7e0f-43d9-4178-807d-c49df48d4f75 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:58:32.791500 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0edf7e0f-43d9-4178-807d-c49df48d4f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:58:32.869919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf7e0f-43d9-4178-807d-c49df48d4f75 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:58:32.874532 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddf573-b179-4aec-8c02-78813a070a1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:58:32.875536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddf573-b179-4aec-8c02-78813a070a1e service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:58:32.875536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddf573-b179-4aec-8c02-78813a070a1e service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:58:32.875722 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddf573-b179-4aec-8c02-78813a070a1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:58:32.877799 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-97ddf573-b179-4aec-8c02-78813a070a1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:58:32.952711 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddf573-b179-4aec-8c02-78813a070a1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:58:32.954143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddf573-b179-4aec-8c02-78813a070a1e service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:58:32.965066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1101/2189] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:58:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:58:34.967119 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094df1c-34fb-42e6-9811-8e963a39a245 service swift] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:58:34.968934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4094df1c-34fb-42e6-9811-8e963a39a245 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:58:35.052459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094df1c-34fb-42e6-9811-8e963a39a245 service swift] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:58:35.056507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246a762-d627-41b8-bd83-9479a5aaeaa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:58:35.057115 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246a762-d627-41b8-bd83-9479a5aaeaa5 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:58:35.057389 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246a762-d627-41b8-bd83-9479a5aaeaa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:58:35.057642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246a762-d627-41b8-bd83-9479a5aaeaa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:58:35.059550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5246a762-d627-41b8-bd83-9479a5aaeaa5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:58:35.075099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246a762-d627-41b8-bd83-9479a5aaeaa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:58:35.076587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246a762-d627-41b8-bd83-9479a5aaeaa5 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:58:35.091821 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1089/2190] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:58:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:58:40.987389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde3f60-2593-4398-99ce-616401230685 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:58:40.987961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde3f60-2593-4398-99ce-616401230685 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:58:40.987961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde3f60-2593-4398-99ce-616401230685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:58:41.010605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-1fde3f60-2593-4398-99ce-616401230685 None None] MFA Rules not processed for user `2f06388453f44fca9e30fbc37da90bba`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:58:41.010605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1fde3f60-2593-4398-99ce-616401230685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:58:41.024406 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1102/2191] 10.222.0.36 () {58 vars in 967 bytes} [Sat Oct 15 13:58:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:58:41.034612 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246a762-d627-41b8-bd83-9479a5aaeaa5 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:58:41.047128 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246a762-d627-41b8-bd83-9479a5aaeaa5 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:58:41.052631 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b92c4-5463-4649-8735-afff833b2bcd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011a40951564323bf7a9c5601df1470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:58:41.053251 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b92c4-5463-4649-8735-afff833b2bcd service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:58:41.053526 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b92c4-5463-4649-8735-afff833b2bcd service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:58:41.053796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b92c4-5463-4649-8735-afff833b2bcd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:58:41.067261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b92c4-5463-4649-8735-afff833b2bcd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:58:41.069973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b92c4-5463-4649-8735-afff833b2bcd service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:58:41.085718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1090/2192] 10.222.0.36 () {60 vars in 1384 bytes} [Sat Oct 15 13:58:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 15 13:58:45.831940 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde3f60-2593-4398-99ce-616401230685 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:58:45.833583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1fde3f60-2593-4398-99ce-616401230685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:58:45.845379 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde3f60-2593-4398-99ce-616401230685 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:58:45.849582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd03614-9ad1-4df0-9a0c-9619de36853c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:58:45.850188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03614-9ad1-4df0-9a0c-9619de36853c service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:58:45.850462 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03614-9ad1-4df0-9a0c-9619de36853c service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:58:45.850730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03614-9ad1-4df0-9a0c-9619de36853c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:58:45.852535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4fd03614-9ad1-4df0-9a0c-9619de36853c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:58:45.924432 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd03614-9ad1-4df0-9a0c-9619de36853c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:58:45.925974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd03614-9ad1-4df0-9a0c-9619de36853c service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:58:45.936244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1103/2193] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:58:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:58:51.077608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=41 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:58:53.732538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292d66-4a4b-4e8a-9149-c66a2af53fbf None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:58:53.733126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292d66-4a4b-4e8a-9149-c66a2af53fbf None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:58:53.733126 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292d66-4a4b-4e8a-9149-c66a2af53fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:58:53.753726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d6292d66-4a4b-4e8a-9149-c66a2af53fbf None None] MFA Rules not processed for user `a10fc1a48c9c41eaa1bf3ff2e4109657`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:58:53.755057 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d6292d66-4a4b-4e8a-9149-c66a2af53fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:58:53.824397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1091/2194] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:58:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:58:54.064514 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd03614-9ad1-4df0-9a0c-9619de36853c service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:58:54.066021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4fd03614-9ad1-4df0-9a0c-9619de36853c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:58:54.079105 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd03614-9ad1-4df0-9a0c-9619de36853c service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:58:54.084477 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d1f4b-6b52-4ddb-9d00-6e15854696ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:58:54.085112 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d1f4b-6b52-4ddb-9d00-6e15854696ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:58:54.085390 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d1f4b-6b52-4ddb-9d00-6e15854696ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:58:54.085657 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d1f4b-6b52-4ddb-9d00-6e15854696ec admin admin] PATH_INFO: `/v3/users/ed6e530c3f544748bd3ffc256b1630b6` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:58:54.086880 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d1f4b-6b52-4ddb-9d00-6e15854696ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed6e530c3f544748bd3ffc256b1630b6)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:58:54.088828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d1f4b-6b52-4ddb-9d00-6e15854696ec admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:58:54.103247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:58:54.157468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-0a7d1f4b-6b52-4ddb-9d00-6e15854696ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed6e530c3f544748bd3ffc256b1630b6'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.165861 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-0a7d1f4b-6b52-4ddb-9d00-6e15854696ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed6e530c3f544748bd3ffc256b1630b6'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.166691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-0a7d1f4b-6b52-4ddb-9d00-6e15854696ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed6e530c3f544748bd3ffc256b1630b6'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.167385 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-0a7d1f4b-6b52-4ddb-9d00-6e15854696ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed6e530c3f544748bd3ffc256b1630b6'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.180750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-0a7d1f4b-6b52-4ddb-9d00-6e15854696ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed6e530c3f544748bd3ffc256b1630b6'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.207578 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1104/2195] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:58:54 2022] DELETE /identity/v3/users/ed6e530c3f544748bd3ffc256b1630b6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:58:54.417867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6292d66-4a4b-4e8a-9149-c66a2af53fbf None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:58:54.441840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6292d66-4a4b-4e8a-9149-c66a2af53fbf None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:58:54.507198 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:58:54.508259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:58:54.508689 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:58:54.508958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] PATH_INFO: `/v3/projects/9a6be8e43c7742be84f4dc3c95a990e5` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:58:54.510092 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a6be8e43c7742be84f4dc3c95a990e5)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:58:54.512085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:58:54.570700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a6be8e43c7742be84f4dc3c95a990e5', 'request_id': 'req-a4b32c59-51d3-4390-a06d-143f9212ea38'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.578582 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a6be8e43c7742be84f4dc3c95a990e5', 'request_id': 'req-a4b32c59-51d3-4390-a06d-143f9212ea38'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.590109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a6be8e43c7742be84f4dc3c95a990e5', 'request_id': 'req-a4b32c59-51d3-4390-a06d-143f9212ea38'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.616501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] The token cache is being invalidate because project 9a6be8e43c7742be84f4dc3c95a990e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:58:54.616811 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.617556 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.619384 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1092/2196] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:58:54 2022] DELETE /identity/v3/projects/9a6be8e43c7742be84f4dc3c95a990e5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:58:54.623826 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d1f4b-6b52-4ddb-9d00-6e15854696ec admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:58:54.625916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0a7d1f4b-6b52-4ddb-9d00-6e15854696ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:58:54.644765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d1f4b-6b52-4ddb-9d00-6e15854696ec admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:58:54.745558 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059efb1-4c91-441a-9c5d-23d3bd65f3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:58:54.746218 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059efb1-4c91-441a-9c5d-23d3bd65f3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:58:54.746616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059efb1-4c91-441a-9c5d-23d3bd65f3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:58:54.746760 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059efb1-4c91-441a-9c5d-23d3bd65f3f8 admin admin] PATH_INFO: `/v3/users/4e1de7598b0f40a7b844879adac1a39b` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:58:54.748054 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059efb1-4c91-441a-9c5d-23d3bd65f3f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e1de7598b0f40a7b844879adac1a39b)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:58:54.750184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059efb1-4c91-441a-9c5d-23d3bd65f3f8 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:58:54.812334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4059efb1-4c91-441a-9c5d-23d3bd65f3f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e1de7598b0f40a7b844879adac1a39b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.819577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4059efb1-4c91-441a-9c5d-23d3bd65f3f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e1de7598b0f40a7b844879adac1a39b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.820364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4059efb1-4c91-441a-9c5d-23d3bd65f3f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e1de7598b0f40a7b844879adac1a39b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.820974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4059efb1-4c91-441a-9c5d-23d3bd65f3f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e1de7598b0f40a7b844879adac1a39b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.830169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4059efb1-4c91-441a-9c5d-23d3bd65f3f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e1de7598b0f40a7b844879adac1a39b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:54.849824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1105/2197] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:58:54 2022] DELETE /identity/v3/users/4e1de7598b0f40a7b844879adac1a39b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:58:55.058977 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:58:55.060726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:58:55.080106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b32c59-51d3-4390-a06d-143f9212ea38 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:58:55.120577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:58:55.148441 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:58:55.149360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:58:55.149645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:58:55.149942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] PATH_INFO: `/v3/projects/d5b8e9a790184f12b5188156c43053a1` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:58:55.151281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5b8e9a790184f12b5188156c43053a1)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:58:55.153687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:58:55.206359 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5b8e9a790184f12b5188156c43053a1', 'request_id': 'req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:55.213185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5b8e9a790184f12b5188156c43053a1', 'request_id': 'req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:55.222989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5b8e9a790184f12b5188156c43053a1', 'request_id': 'req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:55.234863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] The token cache is being invalidate because project d5b8e9a790184f12b5188156c43053a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:58:55.235232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:55.235956 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:58:55.237496 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1093/2198] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:58:55 2022] DELETE /identity/v3/projects/d5b8e9a790184f12b5188156c43053a1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:59:04.845809 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=75,keystone:DELETE=19,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:59:05.231859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=35,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:59:14.056856 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059efb1-4c91-441a-9c5d-23d3bd65f3f8 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:14.058290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4059efb1-4c91-441a-9c5d-23d3bd65f3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:14.119308 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059efb1-4c91-441a-9c5d-23d3bd65f3f8 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:14.173548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be904366-5aa7-4b11-a2b4-9afa778ceba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:14.174071 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be904366-5aa7-4b11-a2b4-9afa778ceba1 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:14.174349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be904366-5aa7-4b11-a2b4-9afa778ceba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:14.174621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be904366-5aa7-4b11-a2b4-9afa778ceba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:14.176601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-be904366-5aa7-4b11-a2b4-9afa778ceba1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:14.242864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be904366-5aa7-4b11-a2b4-9afa778ceba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:14.244340 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be904366-5aa7-4b11-a2b4-9afa778ceba1 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:14.254085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1106/2199] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:59:17.654981 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:17.657166 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:17.671382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee8ef2-a08c-4c7a-8ad9-c216775411e2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:17.675774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c88dd1-bd72-4e5a-95d2-b61b30170e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:17.676513 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c88dd1-bd72-4e5a-95d2-b61b30170e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:17.676870 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c88dd1-bd72-4e5a-95d2-b61b30170e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:17.677197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c88dd1-bd72-4e5a-95d2-b61b30170e8b admin admin] PATH_INFO: `/v3/users/583cce1547144e11978f20a3193ceba4` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:17.692806 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c88dd1-bd72-4e5a-95d2-b61b30170e8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=583cce1547144e11978f20a3193ceba4)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:17.694816 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c88dd1-bd72-4e5a-95d2-b61b30170e8b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:17.754015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-99c88dd1-bd72-4e5a-95d2-b61b30170e8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '583cce1547144e11978f20a3193ceba4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:17.754637 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-99c88dd1-bd72-4e5a-95d2-b61b30170e8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '583cce1547144e11978f20a3193ceba4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:17.755222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-99c88dd1-bd72-4e5a-95d2-b61b30170e8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '583cce1547144e11978f20a3193ceba4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:17.767064 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-99c88dd1-bd72-4e5a-95d2-b61b30170e8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '583cce1547144e11978f20a3193ceba4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:17.776608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-99c88dd1-bd72-4e5a-95d2-b61b30170e8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '583cce1547144e11978f20a3193ceba4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:17.789623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1094/2200] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:59:17 2022] DELETE /identity/v3/users/583cce1547144e11978f20a3193ceba4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:59:17.974715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be904366-5aa7-4b11-a2b4-9afa778ceba1 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:17.991051 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be904366-5aa7-4b11-a2b4-9afa778ceba1 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:18.056404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:18.056953 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:18.057233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:18.057505 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f admin admin] PATH_INFO: `/v3/projects/35f8b5024a194f70a9f7689ac44ab07b` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:18.066450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:59:18.069435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=35f8b5024a194f70a9f7689ac44ab07b)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:18.071818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:18.130337 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35f8b5024a194f70a9f7689ac44ab07b', 'request_id': 'req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:18.139730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35f8b5024a194f70a9f7689ac44ab07b', 'request_id': 'req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:18.147453 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35f8b5024a194f70a9f7689ac44ab07b', 'request_id': 'req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:18.167430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f admin admin] The token cache is being invalidate because project 35f8b5024a194f70a9f7689ac44ab07b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:59:18.167733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:18.168472 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9ef9c020-09a5-4fb2-a365-8acda8f3ae4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:18.170006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1107/2201] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:59:17 2022] DELETE /identity/v3/projects/35f8b5024a194f70a9f7689ac44ab07b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:59:18.175187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738aab58-2e17-4f56-8b70-e3e5fcfc1003 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:18.175468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738aab58-2e17-4f56-8b70-e3e5fcfc1003 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:18.175740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738aab58-2e17-4f56-8b70-e3e5fcfc1003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:18.198340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-738aab58-2e17-4f56-8b70-e3e5fcfc1003 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:59:18.199703 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-738aab58-2e17-4f56-8b70-e3e5fcfc1003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:18.269939 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1095/2202] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:59:18.278714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38619a-2736-465a-a04a-3a4f95a0141c None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:18.279072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38619a-2736-465a-a04a-3a4f95a0141c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:18.279538 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38619a-2736-465a-a04a-3a4f95a0141c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:18.301726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-be38619a-2736-465a-a04a-3a4f95a0141c None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:59:18.303039 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-be38619a-2736-465a-a04a-3a4f95a0141c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:18.312561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1108/2203] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:59:18.318670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738aab58-2e17-4f56-8b70-e3e5fcfc1003 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:18.328945 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738aab58-2e17-4f56-8b70-e3e5fcfc1003 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:18.334002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c960f-ea5c-425f-b533-d3c478b9fe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:18.334603 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c960f-ea5c-425f-b533-d3c478b9fe99 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:18.334863 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c960f-ea5c-425f-b533-d3c478b9fe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:18.335311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c960f-ea5c-425f-b533-d3c478b9fe99 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:18.335897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c960f-ea5c-425f-b533-d3c478b9fe99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:59:18.336234 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c960f-ea5c-425f-b533-d3c478b9fe99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:18.337853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c960f-ea5c-425f-b533-d3c478b9fe99 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:18.352133 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1096/2204] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:59:18.357667 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38619a-2736-465a-a04a-3a4f95a0141c None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:18.371816 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38619a-2736-465a-a04a-3a4f95a0141c None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:18.377286 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cdae5-b17b-4c38-947f-d6758bdf6b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:18.377908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cdae5-b17b-4c38-947f-d6758bdf6b3a admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:18.378190 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cdae5-b17b-4c38-947f-d6758bdf6b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:18.378502 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cdae5-b17b-4c38-947f-d6758bdf6b3a admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:18.379320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484cdae5-b17b-4c38-947f-d6758bdf6b3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:18.382567 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484cdae5-b17b-4c38-947f-d6758bdf6b3a admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:18.429723 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1109/2205] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:59:18 2022] POST /identity/v3/projects => generated 403 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:59:18.432746 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c960f-ea5c-425f-b533-d3c478b9fe99 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:18.443530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c960f-ea5c-425f-b533-d3c478b9fe99 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:18.497155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:59:18.512583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2929645-0f84-4f27-a00d-0ee8edfaaff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:18.513224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2929645-0f84-4f27-a00d-0ee8edfaaff7 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:18.513487 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2929645-0f84-4f27-a00d-0ee8edfaaff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:18.513754 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2929645-0f84-4f27-a00d-0ee8edfaaff7 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:18.514387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2929645-0f84-4f27-a00d-0ee8edfaaff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:18.516355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2929645-0f84-4f27-a00d-0ee8edfaaff7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:18.541314 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1097/2206] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:59:18 2022] POST /identity/v3/users => generated 302 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:59:18.545859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cdae5-b17b-4c38-947f-d6758bdf6b3a admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:18.556650 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484cdae5-b17b-4c38-947f-d6758bdf6b3a admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:18.561576 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84926e-2c36-4d4f-a00c-84788ab1192a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:18.562189 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84926e-2c36-4d4f-a00c-84788ab1192a admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:18.562463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84926e-2c36-4d4f-a00c-84788ab1192a admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:18.562784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84926e-2c36-4d4f-a00c-84788ab1192a admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:18.563377 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84926e-2c36-4d4f-a00c-84788ab1192a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:18.565005 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84926e-2c36-4d4f-a00c-84788ab1192a admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:18.575752 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1110/2207] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:59:18.579989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2929645-0f84-4f27-a00d-0ee8edfaaff7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:18.590616 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2929645-0f84-4f27-a00d-0ee8edfaaff7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:18.595551 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acdf5e-4168-4343-8928-d967da94fa02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:18.596187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acdf5e-4168-4343-8928-d967da94fa02 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:18.596464 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acdf5e-4168-4343-8928-d967da94fa02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:18.596733 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acdf5e-4168-4343-8928-d967da94fa02 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:18.597382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acdf5e-4168-4343-8928-d967da94fa02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:18.600183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acdf5e-4168-4343-8928-d967da94fa02 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:18.614984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1098/2208] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:59:18.620435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84926e-2c36-4d4f-a00c-84788ab1192a admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:18.632628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84926e-2c36-4d4f-a00c-84788ab1192a admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:18.638337 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f491177-cc4c-483a-b5dc-eb70ebc8f91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:18.639003 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f491177-cc4c-483a-b5dc-eb70ebc8f91d admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:18.639278 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f491177-cc4c-483a-b5dc-eb70ebc8f91d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:18.639544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f491177-cc4c-483a-b5dc-eb70ebc8f91d admin admin] PATH_INFO: `/v3/projects/c3e7d3d8d40a4081be69989cecb84ae5/users/8cc8f7e5147c4170b21265e93455813f/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:18.657032 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f491177-cc4c-483a-b5dc-eb70ebc8f91d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e7d3d8d40a4081be69989cecb84ae5, user_id=8cc8f7e5147c4170b21265e93455813f, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:18.659692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f491177-cc4c-483a-b5dc-eb70ebc8f91d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:18.678127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1111/2209] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:59:18 2022] PUT /identity/v3/projects/c3e7d3d8d40a4081be69989cecb84ae5/users/8cc8f7e5147c4170b21265e93455813f/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:59:18.690483 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d28f85-975b-4434-934c-7020fd83ae18 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:18.691075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d28f85-975b-4434-934c-7020fd83ae18 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:18.691075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d28f85-975b-4434-934c-7020fd83ae18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:18.721664 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-37d28f85-975b-4434-934c-7020fd83ae18 None None] MFA Rules not processed for user `8cc8f7e5147c4170b21265e93455813f`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:59:18.722877 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-37d28f85-975b-4434-934c-7020fd83ae18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:18.800765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1099/2210] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:59:18 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:59:18.805946 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f491177-cc4c-483a-b5dc-eb70ebc8f91d admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:18.869097 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f491177-cc4c-483a-b5dc-eb70ebc8f91d admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:18.874525 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902fb52-1ec0-4c00-821a-ad464a60b500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:18.875154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902fb52-1ec0-4c00-821a-ad464a60b500 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:18.875426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902fb52-1ec0-4c00-821a-ad464a60b500 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:18.875667 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902fb52-1ec0-4c00-821a-ad464a60b500 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:18.876345 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902fb52-1ec0-4c00-821a-ad464a60b500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:18.878266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902fb52-1ec0-4c00-821a-ad464a60b500 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:18.902887 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:59:18.911451 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1112/2211] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:59:18 2022] POST /identity/v3/projects => generated 403 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:59:18.916015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d28f85-975b-4434-934c-7020fd83ae18 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:18.927718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d28f85-975b-4434-934c-7020fd83ae18 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:18.997196 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9debd-e5e5-4740-b8af-e4ff828dc83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:18.997824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9debd-e5e5-4740-b8af-e4ff828dc83e admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:18.998853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9debd-e5e5-4740-b8af-e4ff828dc83e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:18.998853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9debd-e5e5-4740-b8af-e4ff828dc83e admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:18.999288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9debd-e5e5-4740-b8af-e4ff828dc83e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:19.001150 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9debd-e5e5-4740-b8af-e4ff828dc83e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:19.027260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1100/2212] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:59:18 2022] POST /identity/v3/users => generated 308 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:59:19.031446 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902fb52-1ec0-4c00-821a-ad464a60b500 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:19.045341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902fb52-1ec0-4c00-821a-ad464a60b500 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:19.050691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970db5a-1a43-464a-a029-26d848fa879f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:19.051337 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970db5a-1a43-464a-a029-26d848fa879f admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:19.051616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970db5a-1a43-464a-a029-26d848fa879f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:19.051905 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970db5a-1a43-464a-a029-26d848fa879f admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:19.052722 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970db5a-1a43-464a-a029-26d848fa879f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:19.054465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970db5a-1a43-464a-a029-26d848fa879f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:19.066690 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1113/2213] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:59:19.071419 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9debd-e5e5-4740-b8af-e4ff828dc83e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:19.082081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9debd-e5e5-4740-b8af-e4ff828dc83e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:19.087160 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076721c-47b1-47a6-ac9a-0fdd70fe1eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:19.087833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076721c-47b1-47a6-ac9a-0fdd70fe1eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:19.088147 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076721c-47b1-47a6-ac9a-0fdd70fe1eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:19.088417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076721c-47b1-47a6-ac9a-0fdd70fe1eeb admin admin] PATH_INFO: `/v3/projects/aa39d76142ad410b85ea2263fb2d4448/users/f71d6d2e1a884149be83d5d6ba682477/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:19.104039 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076721c-47b1-47a6-ac9a-0fdd70fe1eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa39d76142ad410b85ea2263fb2d4448, user_id=f71d6d2e1a884149be83d5d6ba682477, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:19.106725 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076721c-47b1-47a6-ac9a-0fdd70fe1eeb admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:19.123728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1101/2214] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:59:19 2022] PUT /identity/v3/projects/aa39d76142ad410b85ea2263fb2d4448/users/f71d6d2e1a884149be83d5d6ba682477/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:59:19.128196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970db5a-1a43-464a-a029-26d848fa879f admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:19.141663 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970db5a-1a43-464a-a029-26d848fa879f admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:19.213996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369ba39-d654-49ab-9f43-e5077cca7313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:19.214627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369ba39-d654-49ab-9f43-e5077cca7313 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:19.214919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369ba39-d654-49ab-9f43-e5077cca7313 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:19.215190 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369ba39-d654-49ab-9f43-e5077cca7313 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:19.215831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369ba39-d654-49ab-9f43-e5077cca7313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:19.217489 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369ba39-d654-49ab-9f43-e5077cca7313 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:19.228511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1114/2215] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:59:19.233631 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076721c-47b1-47a6-ac9a-0fdd70fe1eeb admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:19.246439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076721c-47b1-47a6-ac9a-0fdd70fe1eeb admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:19.251718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e767c-aa4a-4fd1-a9a0-06402db26214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:19.252546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e767c-aa4a-4fd1-a9a0-06402db26214 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:19.252819 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e767c-aa4a-4fd1-a9a0-06402db26214 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:19.253089 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e767c-aa4a-4fd1-a9a0-06402db26214 admin admin] PATH_INFO: `/v3/projects/aa39d76142ad410b85ea2263fb2d4448/users/f71d6d2e1a884149be83d5d6ba682477/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:19.255117 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e767c-aa4a-4fd1-a9a0-06402db26214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa39d76142ad410b85ea2263fb2d4448, user_id=f71d6d2e1a884149be83d5d6ba682477, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:19.258720 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e767c-aa4a-4fd1-a9a0-06402db26214 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:19.270852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1102/2216] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:59:19 2022] PUT /identity/v3/projects/aa39d76142ad410b85ea2263fb2d4448/users/f71d6d2e1a884149be83d5d6ba682477/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:59:19.282730 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f81233-e548-43cd-bb08-e631278cc48b None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:19.283256 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f81233-e548-43cd-bb08-e631278cc48b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:19.283540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f81233-e548-43cd-bb08-e631278cc48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:19.315708 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-c3f81233-e548-43cd-bb08-e631278cc48b None None] MFA Rules not processed for user `f71d6d2e1a884149be83d5d6ba682477`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:59:19.317374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c3f81233-e548-43cd-bb08-e631278cc48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:19.394326 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1115/2217] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:59:19 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:59:19.405618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e767c-aa4a-4fd1-a9a0-06402db26214 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:19.408113 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fc0e767c-aa4a-4fd1-a9a0-06402db26214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:19.422043 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:59:19.490957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e767c-aa4a-4fd1-a9a0-06402db26214 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:19.495059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4362857-4866-4b84-968c-2db9ace9e695 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:19.495705 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4362857-4866-4b84-968c-2db9ace9e695 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:19.496001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4362857-4866-4b84-968c-2db9ace9e695 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:19.496295 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4362857-4866-4b84-968c-2db9ace9e695 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:19.506591 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4362857-4866-4b84-968c-2db9ace9e695 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:19.508199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4362857-4866-4b84-968c-2db9ace9e695 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:19.518678 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1103/2218] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:59:19 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:59:19.683184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f81233-e548-43cd-bb08-e631278cc48b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:19.751296 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f81233-e548-43cd-bb08-e631278cc48b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:19.755994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963a91b-e28d-4fee-b8da-13111731465f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:19.756637 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963a91b-e28d-4fee-b8da-13111731465f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:19.757176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963a91b-e28d-4fee-b8da-13111731465f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:19.757421 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963a91b-e28d-4fee-b8da-13111731465f admin admin] PATH_INFO: `/v3/users/8cc8f7e5147c4170b21265e93455813f` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:19.758594 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963a91b-e28d-4fee-b8da-13111731465f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cc8f7e5147c4170b21265e93455813f)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:19.760583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963a91b-e28d-4fee-b8da-13111731465f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:19.775237 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:59:19.817856 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4963a91b-e28d-4fee-b8da-13111731465f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cc8f7e5147c4170b21265e93455813f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:19.825385 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4963a91b-e28d-4fee-b8da-13111731465f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cc8f7e5147c4170b21265e93455813f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:19.826132 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4963a91b-e28d-4fee-b8da-13111731465f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cc8f7e5147c4170b21265e93455813f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:19.826772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4963a91b-e28d-4fee-b8da-13111731465f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cc8f7e5147c4170b21265e93455813f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:19.845831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4963a91b-e28d-4fee-b8da-13111731465f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cc8f7e5147c4170b21265e93455813f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:19.866660 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1116/2219] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:59:19 2022] DELETE /identity/v3/users/8cc8f7e5147c4170b21265e93455813f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:59:19.871911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5c653-24b5-4ea7-bcce-548c1f934531 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:19.872488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5c653-24b5-4ea7-bcce-548c1f934531 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:19.872488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5c653-24b5-4ea7-bcce-548c1f934531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:19.892853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-cdc5c653-24b5-4ea7-bcce-548c1f934531 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:59:19.894159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cdc5c653-24b5-4ea7-bcce-548c1f934531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:19.977638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1104/2220] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:59:19.990257 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963a91b-e28d-4fee-b8da-13111731465f admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:19.992124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4963a91b-e28d-4fee-b8da-13111731465f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:20.055288 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963a91b-e28d-4fee-b8da-13111731465f admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:20.059347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1a26b-b4fa-4716-9fba-42f9fcadcd72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:20.060024 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1a26b-b4fa-4716-9fba-42f9fcadcd72 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:20.060304 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1a26b-b4fa-4716-9fba-42f9fcadcd72 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:20.060573 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1a26b-b4fa-4716-9fba-42f9fcadcd72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:20.072688 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1a26b-b4fa-4716-9fba-42f9fcadcd72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:20.074446 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1a26b-b4fa-4716-9fba-42f9fcadcd72 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:20.085478 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1117/2221] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:59:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:59:20.390511 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5c653-24b5-4ea7-bcce-548c1f934531 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:20.392699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cdc5c653-24b5-4ea7-bcce-548c1f934531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:20.409482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5c653-24b5-4ea7-bcce-548c1f934531 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:20.420384 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:20.421073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:20.421342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:20.421606 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] PATH_INFO: `/v3/projects/c3e7d3d8d40a4081be69989cecb84ae5` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:20.422782 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3e7d3d8d40a4081be69989cecb84ae5)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:20.424728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:20.475115 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3e7d3d8d40a4081be69989cecb84ae5', 'request_id': 'req-7a159612-da4d-48a5-ab2b-3de3f867b41c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:20.482358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3e7d3d8d40a4081be69989cecb84ae5', 'request_id': 'req-7a159612-da4d-48a5-ab2b-3de3f867b41c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:20.492334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3e7d3d8d40a4081be69989cecb84ae5', 'request_id': 'req-7a159612-da4d-48a5-ab2b-3de3f867b41c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:20.504079 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] The token cache is being invalidate because project c3e7d3d8d40a4081be69989cecb84ae5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:59:20.504352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:20.505015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:20.506661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1105/2222] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:59:20 2022] DELETE /identity/v3/projects/c3e7d3d8d40a4081be69989cecb84ae5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:59:20.511219 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1a26b-b4fa-4716-9fba-42f9fcadcd72 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:20.524373 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1a26b-b4fa-4716-9fba-42f9fcadcd72 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:20.583306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0426d13-7580-4c75-8f45-8ec7495729d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:20.583986 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0426d13-7580-4c75-8f45-8ec7495729d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:20.584266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0426d13-7580-4c75-8f45-8ec7495729d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:20.584536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0426d13-7580-4c75-8f45-8ec7495729d3 admin admin] PATH_INFO: `/v3/users/f71d6d2e1a884149be83d5d6ba682477` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:20.585872 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0426d13-7580-4c75-8f45-8ec7495729d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f71d6d2e1a884149be83d5d6ba682477)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:20.587801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0426d13-7580-4c75-8f45-8ec7495729d3 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:20.635792 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:59:20.650330 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e0426d13-7580-4c75-8f45-8ec7495729d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f71d6d2e1a884149be83d5d6ba682477'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:20.657792 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e0426d13-7580-4c75-8f45-8ec7495729d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f71d6d2e1a884149be83d5d6ba682477'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:20.658500 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e0426d13-7580-4c75-8f45-8ec7495729d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f71d6d2e1a884149be83d5d6ba682477'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:20.659094 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e0426d13-7580-4c75-8f45-8ec7495729d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f71d6d2e1a884149be83d5d6ba682477'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:20.668682 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-e0426d13-7580-4c75-8f45-8ec7495729d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f71d6d2e1a884149be83d5d6ba682477'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:20.687393 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1118/2223] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 13:59:20 2022] DELETE /identity/v3/users/f71d6d2e1a884149be83d5d6ba682477 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:59:21.014718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:21.016229 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:21.029902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a159612-da4d-48a5-ab2b-3de3f867b41c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:21.067853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:59:21.096189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:21.096839 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:21.097118 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:21.097395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab admin admin] PATH_INFO: `/v3/projects/aa39d76142ad410b85ea2263fb2d4448` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:21.098536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa39d76142ad410b85ea2263fb2d4448)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:21.100730 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:21.157694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa39d76142ad410b85ea2263fb2d4448', 'request_id': 'req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:21.164743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa39d76142ad410b85ea2263fb2d4448', 'request_id': 'req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:21.174468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa39d76142ad410b85ea2263fb2d4448', 'request_id': 'req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:21.186199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab admin admin] The token cache is being invalidate because project aa39d76142ad410b85ea2263fb2d4448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 13:59:21.186485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:21.187127 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bf5364ee-8c0a-4abf-9208-8068aef5e6ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 13:59:21.188732 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1106/2224] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 13:59:21 2022] DELETE /identity/v3/projects/aa39d76142ad410b85ea2263fb2d4448 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:59:21.194128 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7d224-3d51-4077-abd5-52a013e7120f None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:21.194783 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7d224-3d51-4077-abd5-52a013e7120f None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:21.194783 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7d224-3d51-4077-abd5-52a013e7120f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:21.216429 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-b1e7d224-3d51-4077-abd5-52a013e7120f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:59:21.217643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b1e7d224-3d51-4077-abd5-52a013e7120f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:21.281843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1119/2225] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 13:59:21.291305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2acfd-2a18-42bc-bc25-921249bfa806 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:21.291586 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2acfd-2a18-42bc-bc25-921249bfa806 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:21.291860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2acfd-2a18-42bc-bc25-921249bfa806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:21.315271 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-67c2acfd-2a18-42bc-bc25-921249bfa806 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:59:21.316491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-67c2acfd-2a18-42bc-bc25-921249bfa806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:21.324611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1107/2226] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:59:21.329451 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7d224-3d51-4077-abd5-52a013e7120f None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:21.339515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7d224-3d51-4077-abd5-52a013e7120f None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:21.344478 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc276c3e-d820-4beb-b53c-7b7a13057ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:21.345030 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc276c3e-d820-4beb-b53c-7b7a13057ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:21.345353 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc276c3e-d820-4beb-b53c-7b7a13057ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:21.345570 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc276c3e-d820-4beb-b53c-7b7a13057ee2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:21.346203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc276c3e-d820-4beb-b53c-7b7a13057ee2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 13:59:21.346502 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc276c3e-d820-4beb-b53c-7b7a13057ee2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:21.348124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc276c3e-d820-4beb-b53c-7b7a13057ee2 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:21.363445 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1120/2227] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 13:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 13:59:21.369629 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2acfd-2a18-42bc-bc25-921249bfa806 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:21.379411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2acfd-2a18-42bc-bc25-921249bfa806 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:21.384874 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4829662-ac31-4ca6-8552-bacf7a974858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:21.385488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4829662-ac31-4ca6-8552-bacf7a974858 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:21.385764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4829662-ac31-4ca6-8552-bacf7a974858 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:21.386031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4829662-ac31-4ca6-8552-bacf7a974858 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:21.386682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4829662-ac31-4ca6-8552-bacf7a974858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:21.388836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4829662-ac31-4ca6-8552-bacf7a974858 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:21.419375 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1108/2228] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:59:21 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:59:21.423687 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc276c3e-d820-4beb-b53c-7b7a13057ee2 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:21.434208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc276c3e-d820-4beb-b53c-7b7a13057ee2 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:21.498201 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a6894-a461-4f77-87a2-87677d40bee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:21.498824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a6894-a461-4f77-87a2-87677d40bee9 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:21.499107 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a6894-a461-4f77-87a2-87677d40bee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:21.499374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a6894-a461-4f77-87a2-87677d40bee9 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:21.500036 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a6894-a461-4f77-87a2-87677d40bee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:21.502118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a6894-a461-4f77-87a2-87677d40bee9 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:21.527725 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1121/2229] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:59:21 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:59:21.532181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4829662-ac31-4ca6-8552-bacf7a974858 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:21.543072 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4829662-ac31-4ca6-8552-bacf7a974858 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:21.549079 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e41916-7bc7-46b2-b3ac-8ef6c2d6f62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:21.549690 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e41916-7bc7-46b2-b3ac-8ef6c2d6f62f admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:21.549963 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e41916-7bc7-46b2-b3ac-8ef6c2d6f62f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:21.551055 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e41916-7bc7-46b2-b3ac-8ef6c2d6f62f admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:21.551710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e41916-7bc7-46b2-b3ac-8ef6c2d6f62f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:21.553395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e41916-7bc7-46b2-b3ac-8ef6c2d6f62f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:21.566757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1109/2230] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 13:59:21.571551 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a6894-a461-4f77-87a2-87677d40bee9 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:21.585581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a6894-a461-4f77-87a2-87677d40bee9 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:21.592338 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686ad84-05a7-41ad-a577-bc9af7521ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:21.593144 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686ad84-05a7-41ad-a577-bc9af7521ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:21.593519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686ad84-05a7-41ad-a577-bc9af7521ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:21.593920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686ad84-05a7-41ad-a577-bc9af7521ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:21.594526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686ad84-05a7-41ad-a577-bc9af7521ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:21.597108 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686ad84-05a7-41ad-a577-bc9af7521ce5 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:21.613748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1122/2231] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:59:21.618106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e41916-7bc7-46b2-b3ac-8ef6c2d6f62f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:21.628931 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e41916-7bc7-46b2-b3ac-8ef6c2d6f62f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:21.634012 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4564229-7658-427b-9905-3a554f0fb03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:21.634680 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4564229-7658-427b-9905-3a554f0fb03c admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:21.634956 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4564229-7658-427b-9905-3a554f0fb03c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:21.635224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4564229-7658-427b-9905-3a554f0fb03c admin admin] PATH_INFO: `/v3/projects/0b57c243763e4416be29f33b5e76dfec/users/923311fd047b41eaafded69d348a95c3/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:21.652101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4564229-7658-427b-9905-3a554f0fb03c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b57c243763e4416be29f33b5e76dfec, user_id=923311fd047b41eaafded69d348a95c3, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:21.654796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4564229-7658-427b-9905-3a554f0fb03c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:21.670750 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1110/2232] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:59:21 2022] PUT /identity/v3/projects/0b57c243763e4416be29f33b5e76dfec/users/923311fd047b41eaafded69d348a95c3/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:59:21.681677 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3defda-1cc1-47f0-936c-d3760bb2aff8 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:21.681956 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3defda-1cc1-47f0-936c-d3760bb2aff8 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:21.682226 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3defda-1cc1-47f0-936c-d3760bb2aff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:21.713381 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-ed3defda-1cc1-47f0-936c-d3760bb2aff8 None None] MFA Rules not processed for user `923311fd047b41eaafded69d348a95c3`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:59:21.714781 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ed3defda-1cc1-47f0-936c-d3760bb2aff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:21.720848 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:59:21.791693 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1123/2233] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:59:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:59:21.796311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4564229-7658-427b-9905-3a554f0fb03c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:21.860588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4564229-7658-427b-9905-3a554f0fb03c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:21.865405 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324dbcc-7ac4-44e7-ae49-a052d71a7da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:21.866025 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324dbcc-7ac4-44e7-ae49-a052d71a7da8 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:21.866331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324dbcc-7ac4-44e7-ae49-a052d71a7da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:21.866600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324dbcc-7ac4-44e7-ae49-a052d71a7da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:21.867233 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324dbcc-7ac4-44e7-ae49-a052d71a7da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:21.869197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324dbcc-7ac4-44e7-ae49-a052d71a7da8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:21.875860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:59:21.901967 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1111/2234] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 13:59:21 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:59:21.906465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3defda-1cc1-47f0-936c-d3760bb2aff8 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:21.917440 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3defda-1cc1-47f0-936c-d3760bb2aff8 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:21.990061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428ef15-66e1-4d48-a25f-53b47cce1744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:21.990683 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428ef15-66e1-4d48-a25f-53b47cce1744 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:21.990966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428ef15-66e1-4d48-a25f-53b47cce1744 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:21.991332 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428ef15-66e1-4d48-a25f-53b47cce1744 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:21.992021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428ef15-66e1-4d48-a25f-53b47cce1744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:21.993987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428ef15-66e1-4d48-a25f-53b47cce1744 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:22.023244 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1124/2235] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 13:59:21 2022] POST /identity/v3/users => generated 290 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 13:59:22.027260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324dbcc-7ac4-44e7-ae49-a052d71a7da8 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:22.039373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324dbcc-7ac4-44e7-ae49-a052d71a7da8 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:22.045360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384be41-cc2b-4d1f-881c-24f541e28d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:22.046242 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384be41-cc2b-4d1f-881c-24f541e28d8f admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:22.046638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384be41-cc2b-4d1f-881c-24f541e28d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:22.046932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384be41-cc2b-4d1f-881c-24f541e28d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:22.047867 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384be41-cc2b-4d1f-881c-24f541e28d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:22.050242 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384be41-cc2b-4d1f-881c-24f541e28d8f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:22.062130 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1112/2236] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:59:22.066948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428ef15-66e1-4d48-a25f-53b47cce1744 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:22.077590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428ef15-66e1-4d48-a25f-53b47cce1744 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:22.082367 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94ec4-0dbe-41bd-9749-b79b34181d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:22.083030 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94ec4-0dbe-41bd-9749-b79b34181d5e admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:22.083306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94ec4-0dbe-41bd-9749-b79b34181d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:22.083571 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94ec4-0dbe-41bd-9749-b79b34181d5e admin admin] PATH_INFO: `/v3/projects/ea924dde650b4cfc9a26878c0f01c302/users/0b5226b387ef42b09ab2c651d780fc6f/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:22.099566 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd94ec4-0dbe-41bd-9749-b79b34181d5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea924dde650b4cfc9a26878c0f01c302, user_id=0b5226b387ef42b09ab2c651d780fc6f, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:22.102314 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd94ec4-0dbe-41bd-9749-b79b34181d5e admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:22.119956 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1125/2237] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:59:22 2022] PUT /identity/v3/projects/ea924dde650b4cfc9a26878c0f01c302/users/0b5226b387ef42b09ab2c651d780fc6f/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 13:59:22.124297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384be41-cc2b-4d1f-881c-24f541e28d8f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:22.142281 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384be41-cc2b-4d1f-881c-24f541e28d8f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:22.201755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a262d2-1ce0-4b4f-b85c-62aa7a4d6e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:22.202377 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a262d2-1ce0-4b4f-b85c-62aa7a4d6e6d admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:22.202653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a262d2-1ce0-4b4f-b85c-62aa7a4d6e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:22.202922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a262d2-1ce0-4b4f-b85c-62aa7a4d6e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:22.203565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a262d2-1ce0-4b4f-b85c-62aa7a4d6e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:22.205220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a262d2-1ce0-4b4f-b85c-62aa7a4d6e6d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:22.215902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1113/2238] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 13:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 13:59:22.220046 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94ec4-0dbe-41bd-9749-b79b34181d5e admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:22.230838 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94ec4-0dbe-41bd-9749-b79b34181d5e admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:22.238396 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be55e2-9f52-4077-b451-669af062e57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:22.239076 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be55e2-9f52-4077-b451-669af062e57b admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:22.239353 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be55e2-9f52-4077-b451-669af062e57b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:22.239625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be55e2-9f52-4077-b451-669af062e57b admin admin] PATH_INFO: `/v3/projects/ea924dde650b4cfc9a26878c0f01c302/users/0b5226b387ef42b09ab2c651d780fc6f/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:22.241773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be55e2-9f52-4077-b451-669af062e57b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea924dde650b4cfc9a26878c0f01c302, user_id=0b5226b387ef42b09ab2c651d780fc6f, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:22.244444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be55e2-9f52-4077-b451-669af062e57b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:22.255822 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1126/2239] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 13:59:22 2022] PUT /identity/v3/projects/ea924dde650b4cfc9a26878c0f01c302/users/0b5226b387ef42b09ab2c651d780fc6f/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 13:59:22.266774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2412332-a861-4104-9e1a-0beb7d37c89c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:22.267056 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2412332-a861-4104-9e1a-0beb7d37c89c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:22.267354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2412332-a861-4104-9e1a-0beb7d37c89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:22.298811 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f2412332-a861-4104-9e1a-0beb7d37c89c None None] MFA Rules not processed for user `0b5226b387ef42b09ab2c651d780fc6f`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 13:59:22.300098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f2412332-a861-4104-9e1a-0beb7d37c89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:22.374602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1114/2240] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 13:59:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 13:59:22.386315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be55e2-9f52-4077-b451-669af062e57b admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:22.388309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-30be55e2-9f52-4077-b451-669af062e57b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:22.457982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be55e2-9f52-4077-b451-669af062e57b admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:22.463192 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c00fe6-57ed-416c-8f2a-e02502dd7151 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:22.463848 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c00fe6-57ed-416c-8f2a-e02502dd7151 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:22.464143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c00fe6-57ed-416c-8f2a-e02502dd7151 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:22.464406 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c00fe6-57ed-416c-8f2a-e02502dd7151 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:22.474268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c00fe6-57ed-416c-8f2a-e02502dd7151 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:22.475934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c00fe6-57ed-416c-8f2a-e02502dd7151 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:22.487280 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1127/2241] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:59:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:59:22.574004 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2412332-a861-4104-9e1a-0beb7d37c89c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:22.586084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2412332-a861-4104-9e1a-0beb7d37c89c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:22.592084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f0603-2add-45d7-86be-e89207e335c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:22.592949 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f0603-2add-45d7-86be-e89207e335c3 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:22.593289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f0603-2add-45d7-86be-e89207e335c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:22.593670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f0603-2add-45d7-86be-e89207e335c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:22.654131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:59:22.657142 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f0603-2add-45d7-86be-e89207e335c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:22.659175 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f0603-2add-45d7-86be-e89207e335c3 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:22.670246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1115/2242] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:59:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:59:32.481855 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=94,keystone:INSERT=5 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:59:32.666804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 13:59:49.386781 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c00fe6-57ed-416c-8f2a-e02502dd7151 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:49.399345 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c00fe6-57ed-416c-8f2a-e02502dd7151 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:49.403298 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d2d48-a3a8-425f-a086-7ab0edb27287 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:49.403970 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d2d48-a3a8-425f-a086-7ab0edb27287 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:49.404246 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d2d48-a3a8-425f-a086-7ab0edb27287 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:49.404513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d2d48-a3a8-425f-a086-7ab0edb27287 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:49.406479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ed5d2d48-a3a8-425f-a086-7ab0edb27287 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:49.495313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d2d48-a3a8-425f-a086-7ab0edb27287 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:49.496776 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d2d48-a3a8-425f-a086-7ab0edb27287 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:49.515497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1128/2243] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:59:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:59:54.196901 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f0603-2add-45d7-86be-e89207e335c3 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:54.198588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-620f0603-2add-45d7-86be-e89207e335c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:54.258789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f0603-2add-45d7-86be-e89207e335c3 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:54.262547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9627464-5ecc-4c5e-8c60-f4745d63088d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:54.263164 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9627464-5ecc-4c5e-8c60-f4745d63088d service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:54.263459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9627464-5ecc-4c5e-8c60-f4745d63088d service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:54.263713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9627464-5ecc-4c5e-8c60-f4745d63088d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:54.265515 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a9627464-5ecc-4c5e-8c60-f4745d63088d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:54.276164 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9627464-5ecc-4c5e-8c60-f4745d63088d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:54.277536 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9627464-5ecc-4c5e-8c60-f4745d63088d service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:54.285881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1116/2244] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:59:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 13:59:55.756640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d2d48-a3a8-425f-a086-7ab0edb27287 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 13:59:55.759820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ed5d2d48-a3a8-425f-a086-7ab0edb27287 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:55.857096 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d2d48-a3a8-425f-a086-7ab0edb27287 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 13:59:55.863913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d768fc-0eda-4ecc-bada-814553b94ab4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 13:59:55.864486 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d768fc-0eda-4ecc-bada-814553b94ab4 service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 13:59:55.864770 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d768fc-0eda-4ecc-bada-814553b94ab4 service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 13:59:55.865042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d768fc-0eda-4ecc-bada-814553b94ab4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 13:59:55.867408 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-90d768fc-0eda-4ecc-bada-814553b94ab4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 13:59:55.893388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d768fc-0eda-4ecc-bada-814553b94ab4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 13:59:55.894754 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d768fc-0eda-4ecc-bada-814553b94ab4 service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 13:59:55.910180 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1129/2245] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 13:59:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:00.165198 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9627464-5ecc-4c5e-8c60-f4745d63088d service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:00.174151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9627464-5ecc-4c5e-8c60-f4745d63088d service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:00.177917 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b0b06-2b7f-4cd8-8287-c4a4e23fb061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:00.178521 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b0b06-2b7f-4cd8-8287-c4a4e23fb061 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:00.178798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b0b06-2b7f-4cd8-8287-c4a4e23fb061 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:00.179050 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b0b06-2b7f-4cd8-8287-c4a4e23fb061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:00.180895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e67b0b06-2b7f-4cd8-8287-c4a4e23fb061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:00.249558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b0b06-2b7f-4cd8-8287-c4a4e23fb061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:00.251006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b0b06-2b7f-4cd8-8287-c4a4e23fb061 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:00.261203 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1117/2246] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:01.343785 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d768fc-0eda-4ecc-bada-814553b94ab4 service nova] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:01.353331 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d768fc-0eda-4ecc-bada-814553b94ab4 service nova] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:01.357438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3bcae-6a20-4f50-ba61-3d95264f6eb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:01.358050 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3bcae-6a20-4f50-ba61-3d95264f6eb3 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:01.358323 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3bcae-6a20-4f50-ba61-3d95264f6eb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:01.358596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3bcae-6a20-4f50-ba61-3d95264f6eb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:01.360397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-fbc3bcae-6a20-4f50-ba61-3d95264f6eb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:01.430330 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:01.433560 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3bcae-6a20-4f50-ba61-3d95264f6eb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:01.435641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3bcae-6a20-4f50-ba61-3d95264f6eb3 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:01.446352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1130/2247] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:03.631222 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b0b06-2b7f-4cd8-8287-c4a4e23fb061 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:03.632799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e67b0b06-2b7f-4cd8-8287-c4a4e23fb061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:03.646206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b0b06-2b7f-4cd8-8287-c4a4e23fb061 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:03.652318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3beca27-519e-4e4b-9d58-6bdbdd7fb28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:03.653087 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3beca27-519e-4e4b-9d58-6bdbdd7fb28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:03.653362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3beca27-519e-4e4b-9d58-6bdbdd7fb28d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:03.653629 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3beca27-519e-4e4b-9d58-6bdbdd7fb28d admin admin] PATH_INFO: `/v3/users/0b01e277944d4edcb8625ebf4544e340` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:03.654988 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3beca27-519e-4e4b-9d58-6bdbdd7fb28d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b01e277944d4edcb8625ebf4544e340)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:03.657446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3beca27-519e-4e4b-9d58-6bdbdd7fb28d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:03.710810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d3beca27-519e-4e4b-9d58-6bdbdd7fb28d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b01e277944d4edcb8625ebf4544e340'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:03.711485 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d3beca27-519e-4e4b-9d58-6bdbdd7fb28d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b01e277944d4edcb8625ebf4544e340'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:03.712471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d3beca27-519e-4e4b-9d58-6bdbdd7fb28d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b01e277944d4edcb8625ebf4544e340'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:03.725918 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d3beca27-519e-4e4b-9d58-6bdbdd7fb28d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b01e277944d4edcb8625ebf4544e340'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:03.735737 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d3beca27-519e-4e4b-9d58-6bdbdd7fb28d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b01e277944d4edcb8625ebf4544e340'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:03.749506 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1118/2248] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:03 2022] DELETE /identity/v3/users/0b01e277944d4edcb8625ebf4544e340 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:03.938535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3bcae-6a20-4f50-ba61-3d95264f6eb3 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:03.954666 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3bcae-6a20-4f50-ba61-3d95264f6eb3 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:04.019534 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:04.020201 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:04.020474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:04.020763 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce admin admin] PATH_INFO: `/v3/projects/aabf1193b43040948317be442c5c16f7` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:04.021925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=aabf1193b43040948317be442c5c16f7)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:04.023853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:04.073451 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aabf1193b43040948317be442c5c16f7', 'request_id': 'req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:04.082963 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aabf1193b43040948317be442c5c16f7', 'request_id': 'req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:04.089942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aabf1193b43040948317be442c5c16f7', 'request_id': 'req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:04.100686 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce admin admin] The token cache is being invalidate because project aabf1193b43040948317be442c5c16f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:04.100972 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:04.101684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-21fdd7a2-bc59-4e9b-b135-5b48a0f0a6ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:04.103230 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1131/2249] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:03 2022] DELETE /identity/v3/projects/aabf1193b43040948317be442c5c16f7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:04.108709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bd4db-e7ea-4acf-98bd-f872d9c6536d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:04.108992 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bd4db-e7ea-4acf-98bd-f872d9c6536d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:04.109259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bd4db-e7ea-4acf-98bd-f872d9c6536d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:04.129463 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b81bd4db-e7ea-4acf-98bd-f872d9c6536d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:04.130650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b81bd4db-e7ea-4acf-98bd-f872d9c6536d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:04.135172 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:04.196948 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1119/2250] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 14:00:04.205222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9d29e-2f18-4ce8-8970-10ef906089fd None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:04.205581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9d29e-2f18-4ce8-8970-10ef906089fd None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:04.205857 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9d29e-2f18-4ce8-8970-10ef906089fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:04.226473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-c6a9d29e-2f18-4ce8-8970-10ef906089fd None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:04.227734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c6a9d29e-2f18-4ce8-8970-10ef906089fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:04.236651 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1132/2251] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:04.241014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81bd4db-e7ea-4acf-98bd-f872d9c6536d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:04.253303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81bd4db-e7ea-4acf-98bd-f872d9c6536d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:04.258897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d354d-2600-47ba-92ed-f1bef689de7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:04.259567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d354d-2600-47ba-92ed-f1bef689de7d admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:04.259904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d354d-2600-47ba-92ed-f1bef689de7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:04.260174 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d354d-2600-47ba-92ed-f1bef689de7d admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:04.260810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d354d-2600-47ba-92ed-f1bef689de7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:00:04.261109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d354d-2600-47ba-92ed-f1bef689de7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:04.262725 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d354d-2600-47ba-92ed-f1bef689de7d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:04.275282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1120/2252] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 14:00:04.280029 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9d29e-2f18-4ce8-8970-10ef906089fd None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:04.294336 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9d29e-2f18-4ce8-8970-10ef906089fd None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:04.301256 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02015f-02ad-4dc1-83bf-d08b0f89a25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:04.302138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02015f-02ad-4dc1-83bf-d08b0f89a25b admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:04.302415 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02015f-02ad-4dc1-83bf-d08b0f89a25b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:04.302685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02015f-02ad-4dc1-83bf-d08b0f89a25b admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:04.303576 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b02015f-02ad-4dc1-83bf-d08b0f89a25b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:04.306521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b02015f-02ad-4dc1-83bf-d08b0f89a25b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:04.342957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1133/2253] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:04 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:04.349325 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d354d-2600-47ba-92ed-f1bef689de7d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:04.365586 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d354d-2600-47ba-92ed-f1bef689de7d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:04.432953 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2114e0d-1b49-47d2-8daf-158ba791b4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:04.433774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2114e0d-1b49-47d2-8daf-158ba791b4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:04.434116 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2114e0d-1b49-47d2-8daf-158ba791b4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:04.434475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2114e0d-1b49-47d2-8daf-158ba791b4c4 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:04.435357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2114e0d-1b49-47d2-8daf-158ba791b4c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:04.438322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2114e0d-1b49-47d2-8daf-158ba791b4c4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:04.471680 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1121/2254] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:04 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:04.476031 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02015f-02ad-4dc1-83bf-d08b0f89a25b admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:04.487644 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02015f-02ad-4dc1-83bf-d08b0f89a25b admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:04.493424 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ee814-4123-40e9-bef5-ab42313c2bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:04.494044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ee814-4123-40e9-bef5-ab42313c2bde admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:04.494325 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ee814-4123-40e9-bef5-ab42313c2bde admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:04.494589 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ee814-4123-40e9-bef5-ab42313c2bde admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:04.495238 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ee814-4123-40e9-bef5-ab42313c2bde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:04.496894 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ee814-4123-40e9-bef5-ab42313c2bde admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:04.507771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1134/2255] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:04.512207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2114e0d-1b49-47d2-8daf-158ba791b4c4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:04.526197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2114e0d-1b49-47d2-8daf-158ba791b4c4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:04.532390 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725d057-518a-4c1c-81d3-8dc4f0a89d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:04.533178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725d057-518a-4c1c-81d3-8dc4f0a89d9b admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:04.533657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725d057-518a-4c1c-81d3-8dc4f0a89d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:04.533805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725d057-518a-4c1c-81d3-8dc4f0a89d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:04.534769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725d057-518a-4c1c-81d3-8dc4f0a89d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:04.537215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725d057-518a-4c1c-81d3-8dc4f0a89d9b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:04.553558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1122/2256] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:04.558819 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ee814-4123-40e9-bef5-ab42313c2bde admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:04.568833 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ee814-4123-40e9-bef5-ab42313c2bde admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:04.574442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaad34d-d13b-4d96-a1a1-fd88cf4e6426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:04.575088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaad34d-d13b-4d96-a1a1-fd88cf4e6426 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:04.575362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaad34d-d13b-4d96-a1a1-fd88cf4e6426 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:04.575629 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaad34d-d13b-4d96-a1a1-fd88cf4e6426 admin admin] PATH_INFO: `/v3/projects/d978d566487c4f82a30af41db42fc2b8/users/236b7010a9554b099c15f05baa4405d8/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:04.592835 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaad34d-d13b-4d96-a1a1-fd88cf4e6426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d978d566487c4f82a30af41db42fc2b8, user_id=236b7010a9554b099c15f05baa4405d8, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:04.595540 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaad34d-d13b-4d96-a1a1-fd88cf4e6426 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:04.613406 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1135/2257] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:04 2022] PUT /identity/v3/projects/d978d566487c4f82a30af41db42fc2b8/users/236b7010a9554b099c15f05baa4405d8/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:04.626210 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67efb-c9f8-45b7-9c2c-667abb253cb0 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:04.626504 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67efb-c9f8-45b7-9c2c-667abb253cb0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:04.626853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67efb-c9f8-45b7-9c2c-667abb253cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:04.658345 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b7e67efb-c9f8-45b7-9c2c-667abb253cb0 None None] MFA Rules not processed for user `236b7010a9554b099c15f05baa4405d8`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:04.659542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b7e67efb-c9f8-45b7-9c2c-667abb253cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:04.727617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1123/2258] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:04.732805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaad34d-d13b-4d96-a1a1-fd88cf4e6426 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:04.755742 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:04.802058 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaad34d-d13b-4d96-a1a1-fd88cf4e6426 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:04.808123 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a9706-ffab-4fd1-b4f2-d41dc2fe07fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:04.808750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a9706-ffab-4fd1-b4f2-d41dc2fe07fa admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:04.809051 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a9706-ffab-4fd1-b4f2-d41dc2fe07fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:04.809316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a9706-ffab-4fd1-b4f2-d41dc2fe07fa admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:04.809943 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a9706-ffab-4fd1-b4f2-d41dc2fe07fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:04.811928 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a9706-ffab-4fd1-b4f2-d41dc2fe07fa admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:04.847011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1136/2259] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:04 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:04.852074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e67efb-c9f8-45b7-9c2c-667abb253cb0 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:04.860449 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:04.865047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e67efb-c9f8-45b7-9c2c-667abb253cb0 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:04.928711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0047e-290e-4999-a15d-c8eb35b6b566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:04.929311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0047e-290e-4999-a15d-c8eb35b6b566 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:04.929580 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0047e-290e-4999-a15d-c8eb35b6b566 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:04.929861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0047e-290e-4999-a15d-c8eb35b6b566 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:04.930507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0047e-290e-4999-a15d-c8eb35b6b566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:04.932727 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0047e-290e-4999-a15d-c8eb35b6b566 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:04.957628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1124/2260] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:04 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:04.961564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a9706-ffab-4fd1-b4f2-d41dc2fe07fa admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:04.974734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a9706-ffab-4fd1-b4f2-d41dc2fe07fa admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:04.986958 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891143b-0c45-4930-997f-13444e008bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:04.986958 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891143b-0c45-4930-997f-13444e008bca admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:04.986958 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891143b-0c45-4930-997f-13444e008bca admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:04.986958 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891143b-0c45-4930-997f-13444e008bca admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:04.987600 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891143b-0c45-4930-997f-13444e008bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:04.987600 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891143b-0c45-4930-997f-13444e008bca admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:05.003140 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1137/2261] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:05.007755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0047e-290e-4999-a15d-c8eb35b6b566 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:05.024395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0047e-290e-4999-a15d-c8eb35b6b566 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:05.044310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a83ef8-dac3-4828-98fa-d93f0b174f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:05.045232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a83ef8-dac3-4828-98fa-d93f0b174f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:05.045510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a83ef8-dac3-4828-98fa-d93f0b174f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:05.045777 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a83ef8-dac3-4828-98fa-d93f0b174f5f admin admin] PATH_INFO: `/v3/projects/85ca2a8f228446adb78d667d55755ae6/users/2b04ff43da104b57940d5cbaad936878/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:05.062471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a83ef8-dac3-4828-98fa-d93f0b174f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ca2a8f228446adb78d667d55755ae6, user_id=2b04ff43da104b57940d5cbaad936878, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:05.066597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a83ef8-dac3-4828-98fa-d93f0b174f5f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:05.083660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1125/2262] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:05 2022] PUT /identity/v3/projects/85ca2a8f228446adb78d667d55755ae6/users/2b04ff43da104b57940d5cbaad936878/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:05.088088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891143b-0c45-4930-997f-13444e008bca admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:05.098587 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891143b-0c45-4930-997f-13444e008bca admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:05.158415 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38593fd1-96b9-43b6-b08b-e67b33362d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:05.159110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38593fd1-96b9-43b6-b08b-e67b33362d32 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:05.159373 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38593fd1-96b9-43b6-b08b-e67b33362d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:05.159651 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38593fd1-96b9-43b6-b08b-e67b33362d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:05.160326 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38593fd1-96b9-43b6-b08b-e67b33362d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:05.162007 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38593fd1-96b9-43b6-b08b-e67b33362d32 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:05.172474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1138/2263] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:05.177242 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a83ef8-dac3-4828-98fa-d93f0b174f5f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:05.189437 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a83ef8-dac3-4828-98fa-d93f0b174f5f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:05.194199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce3618-5703-421e-a698-88e840b18621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:05.194862 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce3618-5703-421e-a698-88e840b18621 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:05.195335 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce3618-5703-421e-a698-88e840b18621 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:05.195446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce3618-5703-421e-a698-88e840b18621 admin admin] PATH_INFO: `/v3/projects/85ca2a8f228446adb78d667d55755ae6/users/2b04ff43da104b57940d5cbaad936878/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:05.197539 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce3618-5703-421e-a698-88e840b18621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ca2a8f228446adb78d667d55755ae6, user_id=2b04ff43da104b57940d5cbaad936878, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:05.200234 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce3618-5703-421e-a698-88e840b18621 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:05.211055 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1126/2264] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:05 2022] PUT /identity/v3/projects/85ca2a8f228446adb78d667d55755ae6/users/2b04ff43da104b57940d5cbaad936878/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:05.221777 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fb4bd-06a6-468c-89f1-f71a3090eb8b None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:05.222061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fb4bd-06a6-468c-89f1-f71a3090eb8b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:05.222337 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fb4bd-06a6-468c-89f1-f71a3090eb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:05.252091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-254fb4bd-06a6-468c-89f1-f71a3090eb8b None None] MFA Rules not processed for user `2b04ff43da104b57940d5cbaad936878`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:05.253332 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-254fb4bd-06a6-468c-89f1-f71a3090eb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:05.328166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1139/2265] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:05.342362 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce3618-5703-421e-a698-88e840b18621 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:05.344617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-24ce3618-5703-421e-a698-88e840b18621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:05.403375 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce3618-5703-421e-a698-88e840b18621 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:05.407286 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55f619-2fbe-4838-bb08-412f1a377374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:05.407894 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55f619-2fbe-4838-bb08-412f1a377374 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:05.408190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55f619-2fbe-4838-bb08-412f1a377374 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:05.408453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55f619-2fbe-4838-bb08-412f1a377374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:05.418613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55f619-2fbe-4838-bb08-412f1a377374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:05.420087 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55f619-2fbe-4838-bb08-412f1a377374 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:05.430009 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1127/2266] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:05.877841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fb4bd-06a6-468c-89f1-f71a3090eb8b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:05.898144 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:05.944941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fb4bd-06a6-468c-89f1-f71a3090eb8b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:05.949807 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14e9f9-b8f5-4153-9231-d145cf416760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:05.950444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14e9f9-b8f5-4153-9231-d145cf416760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:05.950720 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14e9f9-b8f5-4153-9231-d145cf416760 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:05.950990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14e9f9-b8f5-4153-9231-d145cf416760 admin admin] PATH_INFO: `/v3/users/236b7010a9554b099c15f05baa4405d8` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:05.952257 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14e9f9-b8f5-4153-9231-d145cf416760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=236b7010a9554b099c15f05baa4405d8)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:05.954182 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14e9f9-b8f5-4153-9231-d145cf416760 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:06.011149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ba14e9f9-b8f5-4153-9231-d145cf416760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '236b7010a9554b099c15f05baa4405d8'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.019439 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ba14e9f9-b8f5-4153-9231-d145cf416760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '236b7010a9554b099c15f05baa4405d8'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.020187 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ba14e9f9-b8f5-4153-9231-d145cf416760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '236b7010a9554b099c15f05baa4405d8'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.020845 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ba14e9f9-b8f5-4153-9231-d145cf416760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '236b7010a9554b099c15f05baa4405d8'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.030805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ba14e9f9-b8f5-4153-9231-d145cf416760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '236b7010a9554b099c15f05baa4405d8'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.052937 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1140/2267] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:05 2022] DELETE /identity/v3/users/236b7010a9554b099c15f05baa4405d8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:06.058587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb7d21-af8e-43ff-9b40-b5374e171104 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:06.059110 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb7d21-af8e-43ff-9b40-b5374e171104 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:06.059212 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb7d21-af8e-43ff-9b40-b5374e171104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:06.081991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f7bb7d21-af8e-43ff-9b40-b5374e171104 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:06.083282 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f7bb7d21-af8e-43ff-9b40-b5374e171104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:06.135965 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:06.145586 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1128/2268] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:06.156167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14e9f9-b8f5-4153-9231-d145cf416760 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:06.158254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ba14e9f9-b8f5-4153-9231-d145cf416760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:06.225908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14e9f9-b8f5-4153-9231-d145cf416760 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:06.231661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd91d8-c04b-479e-bfb7-f496857b493f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:06.232319 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd91d8-c04b-479e-bfb7-f496857b493f service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:06.232598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd91d8-c04b-479e-bfb7-f496857b493f service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:06.232866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd91d8-c04b-479e-bfb7-f496857b493f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:06.244014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd91d8-c04b-479e-bfb7-f496857b493f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:06.245770 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd91d8-c04b-479e-bfb7-f496857b493f service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:06.255910 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1141/2269] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:06.570127 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb7d21-af8e-43ff-9b40-b5374e171104 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:06.571764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f7bb7d21-af8e-43ff-9b40-b5374e171104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:06.584609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb7d21-af8e-43ff-9b40-b5374e171104 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:06.590662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:06.591193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:06.591469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:06.591712 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] PATH_INFO: `/v3/projects/d978d566487c4f82a30af41db42fc2b8` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:06.593285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d978d566487c4f82a30af41db42fc2b8)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:06.595302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:06.646350 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd978d566487c4f82a30af41db42fc2b8', 'request_id': 'req-b75457cf-88f1-4030-a316-07c7c29d5bb6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.652850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd978d566487c4f82a30af41db42fc2b8', 'request_id': 'req-b75457cf-88f1-4030-a316-07c7c29d5bb6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.663077 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd978d566487c4f82a30af41db42fc2b8', 'request_id': 'req-b75457cf-88f1-4030-a316-07c7c29d5bb6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.674598 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] The token cache is being invalidate because project d978d566487c4f82a30af41db42fc2b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:06.674882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.675524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.677074 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1129/2270] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:06 2022] DELETE /identity/v3/projects/d978d566487c4f82a30af41db42fc2b8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:06.681329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd91d8-c04b-479e-bfb7-f496857b493f service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:06.691373 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd91d8-c04b-479e-bfb7-f496857b493f service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:06.725789 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:06.750729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bdf15-12e2-469c-8c56-8430db101d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:06.751292 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bdf15-12e2-469c-8c56-8430db101d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:06.751567 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bdf15-12e2-469c-8c56-8430db101d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:06.751848 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bdf15-12e2-469c-8c56-8430db101d17 admin admin] PATH_INFO: `/v3/users/2b04ff43da104b57940d5cbaad936878` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:06.752985 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bdf15-12e2-469c-8c56-8430db101d17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b04ff43da104b57940d5cbaad936878)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:06.754919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bdf15-12e2-469c-8c56-8430db101d17 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:06.809592 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a79bdf15-12e2-469c-8c56-8430db101d17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b04ff43da104b57940d5cbaad936878'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.818501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a79bdf15-12e2-469c-8c56-8430db101d17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b04ff43da104b57940d5cbaad936878'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.819577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a79bdf15-12e2-469c-8c56-8430db101d17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b04ff43da104b57940d5cbaad936878'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.820627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a79bdf15-12e2-469c-8c56-8430db101d17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b04ff43da104b57940d5cbaad936878'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.831011 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a79bdf15-12e2-469c-8c56-8430db101d17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b04ff43da104b57940d5cbaad936878'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:06.852941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1142/2271] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:06 2022] DELETE /identity/v3/users/2b04ff43da104b57940d5cbaad936878 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:07.141760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:07.143382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:07.157921 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75457cf-88f1-4030-a316-07c7c29d5bb6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:07.217917 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c305b-9791-4608-ab75-e4910fae4d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:07.218550 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c305b-9791-4608-ab75-e4910fae4d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:07.218806 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c305b-9791-4608-ab75-e4910fae4d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:07.219089 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c305b-9791-4608-ab75-e4910fae4d2b admin admin] PATH_INFO: `/v3/projects/85ca2a8f228446adb78d667d55755ae6` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:07.220237 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c305b-9791-4608-ab75-e4910fae4d2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=85ca2a8f228446adb78d667d55755ae6)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:07.222127 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c305b-9791-4608-ab75-e4910fae4d2b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:07.273256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-554c305b-9791-4608-ab75-e4910fae4d2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85ca2a8f228446adb78d667d55755ae6', 'request_id': 'req-554c305b-9791-4608-ab75-e4910fae4d2b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:07.280809 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-554c305b-9791-4608-ab75-e4910fae4d2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85ca2a8f228446adb78d667d55755ae6', 'request_id': 'req-554c305b-9791-4608-ab75-e4910fae4d2b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:07.290390 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-554c305b-9791-4608-ab75-e4910fae4d2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85ca2a8f228446adb78d667d55755ae6', 'request_id': 'req-554c305b-9791-4608-ab75-e4910fae4d2b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:07.301518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-554c305b-9791-4608-ab75-e4910fae4d2b admin admin] The token cache is being invalidate because project 85ca2a8f228446adb78d667d55755ae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:07.301805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-554c305b-9791-4608-ab75-e4910fae4d2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:07.302482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-554c305b-9791-4608-ab75-e4910fae4d2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:07.304032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1130/2272] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:07 2022] DELETE /identity/v3/projects/85ca2a8f228446adb78d667d55755ae6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:07.309351 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a401a-eac2-4779-911c-5da0ad149b20 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:07.309833 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a401a-eac2-4779-911c-5da0ad149b20 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:07.309929 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a401a-eac2-4779-911c-5da0ad149b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:07.329826 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-587a401a-eac2-4779-911c-5da0ad149b20 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:07.331041 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-587a401a-eac2-4779-911c-5da0ad149b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:07.395149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1143/2273] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:07.403411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12edf9-1115-4232-9975-1b6d768a688a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:07.403620 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12edf9-1115-4232-9975-1b6d768a688a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:07.403892 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12edf9-1115-4232-9975-1b6d768a688a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:07.424874 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9f12edf9-1115-4232-9975-1b6d768a688a None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:07.426058 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9f12edf9-1115-4232-9975-1b6d768a688a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:07.434483 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1131/2274] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:07.439483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a401a-eac2-4779-911c-5da0ad149b20 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:07.456269 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a401a-eac2-4779-911c-5da0ad149b20 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:07.463780 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec73ded-0333-43d7-91bf-05082c1baabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:07.464422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec73ded-0333-43d7-91bf-05082c1baabc admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:07.464700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec73ded-0333-43d7-91bf-05082c1baabc admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:07.464968 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec73ded-0333-43d7-91bf-05082c1baabc admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:07.465626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec73ded-0333-43d7-91bf-05082c1baabc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:00:07.465928 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec73ded-0333-43d7-91bf-05082c1baabc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:07.467543 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec73ded-0333-43d7-91bf-05082c1baabc admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:07.480616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1144/2275] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 14:00:07.484957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12edf9-1115-4232-9975-1b6d768a688a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:07.494801 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12edf9-1115-4232-9975-1b6d768a688a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:07.500507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14430672-89c4-4ed7-a6f1-8b03f84b9dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:07.501094 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14430672-89c4-4ed7-a6f1-8b03f84b9dff admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:07.501392 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14430672-89c4-4ed7-a6f1-8b03f84b9dff admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:07.501657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14430672-89c4-4ed7-a6f1-8b03f84b9dff admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:07.502288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14430672-89c4-4ed7-a6f1-8b03f84b9dff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:07.504242 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14430672-89c4-4ed7-a6f1-8b03f84b9dff admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:07.509065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:07.537629 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1132/2276] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:07 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:07.544154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec73ded-0333-43d7-91bf-05082c1baabc admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:07.555357 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec73ded-0333-43d7-91bf-05082c1baabc admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:07.599019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:07.616882 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16103c81-d73f-49b6-9f5e-cd3cf07f6990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:07.617463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16103c81-d73f-49b6-9f5e-cd3cf07f6990 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:07.617743 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16103c81-d73f-49b6-9f5e-cd3cf07f6990 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:07.618006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16103c81-d73f-49b6-9f5e-cd3cf07f6990 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:07.618639 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16103c81-d73f-49b6-9f5e-cd3cf07f6990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:07.620772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16103c81-d73f-49b6-9f5e-cd3cf07f6990 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:07.645380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1145/2277] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:07 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:07.649810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14430672-89c4-4ed7-a6f1-8b03f84b9dff admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:07.660632 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14430672-89c4-4ed7-a6f1-8b03f84b9dff admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:07.666759 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59744cb9-40cf-4cce-ac22-5c109d190fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:07.667360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59744cb9-40cf-4cce-ac22-5c109d190fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:07.667662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59744cb9-40cf-4cce-ac22-5c109d190fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:07.667940 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59744cb9-40cf-4cce-ac22-5c109d190fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:07.668595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59744cb9-40cf-4cce-ac22-5c109d190fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:07.675179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59744cb9-40cf-4cce-ac22-5c109d190fb2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:07.681370 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1133/2278] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:07.686091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16103c81-d73f-49b6-9f5e-cd3cf07f6990 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:07.700576 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16103c81-d73f-49b6-9f5e-cd3cf07f6990 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:07.706015 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56c780-9050-4ae6-9d6d-d344e5a79b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:07.706636 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56c780-9050-4ae6-9d6d-d344e5a79b7d admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:07.706907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56c780-9050-4ae6-9d6d-d344e5a79b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:07.707175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56c780-9050-4ae6-9d6d-d344e5a79b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:07.707822 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56c780-9050-4ae6-9d6d-d344e5a79b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:07.709474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c56c780-9050-4ae6-9d6d-d344e5a79b7d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:07.721012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1146/2279] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:07.725512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59744cb9-40cf-4cce-ac22-5c109d190fb2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:07.736509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59744cb9-40cf-4cce-ac22-5c109d190fb2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:07.742231 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae2e84-cb61-4aaa-95b2-a2554da8667f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:07.742881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae2e84-cb61-4aaa-95b2-a2554da8667f admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:07.743151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae2e84-cb61-4aaa-95b2-a2554da8667f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:07.743415 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae2e84-cb61-4aaa-95b2-a2554da8667f admin admin] PATH_INFO: `/v3/projects/fa5684fd117b4e06979f0c2afbe5006c/users/9db9b9d087674632a22beaf010b84688/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:07.759398 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae2e84-cb61-4aaa-95b2-a2554da8667f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa5684fd117b4e06979f0c2afbe5006c, user_id=9db9b9d087674632a22beaf010b84688, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:07.762156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae2e84-cb61-4aaa-95b2-a2554da8667f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:07.779465 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1134/2280] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:07 2022] PUT /identity/v3/projects/fa5684fd117b4e06979f0c2afbe5006c/users/9db9b9d087674632a22beaf010b84688/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:07.790612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473c1a7-6cd5-4555-8491-1527264265c7 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:07.790888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473c1a7-6cd5-4555-8491-1527264265c7 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:07.791200 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473c1a7-6cd5-4555-8491-1527264265c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:07.822247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-c473c1a7-6cd5-4555-8491-1527264265c7 None None] MFA Rules not processed for user `9db9b9d087674632a22beaf010b84688`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:07.823626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c473c1a7-6cd5-4555-8491-1527264265c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:07.896685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1147/2281] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:07.901452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae2e84-cb61-4aaa-95b2-a2554da8667f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:07.969878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae2e84-cb61-4aaa-95b2-a2554da8667f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:07.975302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599e01b-7766-443c-b5e4-068857192f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:07.975907 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599e01b-7766-443c-b5e4-068857192f87 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:07.976215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599e01b-7766-443c-b5e4-068857192f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:07.976488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599e01b-7766-443c-b5e4-068857192f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:07.977114 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599e01b-7766-443c-b5e4-068857192f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:07.979057 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599e01b-7766-443c-b5e4-068857192f87 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:08.009869 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1135/2282] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:07 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:08.014566 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473c1a7-6cd5-4555-8491-1527264265c7 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:08.029505 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473c1a7-6cd5-4555-8491-1527264265c7 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:08.094262 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d40c22-a4d8-499e-b44d-7bf7af7760f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:08.094744 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d40c22-a4d8-499e-b44d-7bf7af7760f8 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:08.095032 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d40c22-a4d8-499e-b44d-7bf7af7760f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:08.095302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d40c22-a4d8-499e-b44d-7bf7af7760f8 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:08.095962 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d40c22-a4d8-499e-b44d-7bf7af7760f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:08.097894 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d40c22-a4d8-499e-b44d-7bf7af7760f8 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:08.121970 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1148/2283] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:08 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 14:00:08.126128 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599e01b-7766-443c-b5e4-068857192f87 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:08.136406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599e01b-7766-443c-b5e4-068857192f87 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:08.141041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338b620-d11c-428c-a51a-eb15ab33022f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:08.141632 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338b620-d11c-428c-a51a-eb15ab33022f admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:08.141903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338b620-d11c-428c-a51a-eb15ab33022f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:08.142454 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338b620-d11c-428c-a51a-eb15ab33022f admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:08.142783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338b620-d11c-428c-a51a-eb15ab33022f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:08.144419 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338b620-d11c-428c-a51a-eb15ab33022f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:08.155011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1136/2284] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:08.159212 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d40c22-a4d8-499e-b44d-7bf7af7760f8 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:08.169957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d40c22-a4d8-499e-b44d-7bf7af7760f8 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:08.174940 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec2eae-c6f1-4312-8dd5-e52e871ac919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:08.175591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec2eae-c6f1-4312-8dd5-e52e871ac919 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:08.175920 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec2eae-c6f1-4312-8dd5-e52e871ac919 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:08.176195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec2eae-c6f1-4312-8dd5-e52e871ac919 admin admin] PATH_INFO: `/v3/projects/a454348c912a40b3b0342137d02793e2/users/e0271ae1e5b84ed78af71dce187bb692/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:08.191912 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec2eae-c6f1-4312-8dd5-e52e871ac919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a454348c912a40b3b0342137d02793e2, user_id=e0271ae1e5b84ed78af71dce187bb692, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:08.195015 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec2eae-c6f1-4312-8dd5-e52e871ac919 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:08.210067 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1149/2285] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:08 2022] PUT /identity/v3/projects/a454348c912a40b3b0342137d02793e2/users/e0271ae1e5b84ed78af71dce187bb692/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:08.214305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338b620-d11c-428c-a51a-eb15ab33022f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:08.224600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338b620-d11c-428c-a51a-eb15ab33022f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:08.286642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531726c6-5f91-49a8-b109-0aded074ed0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:08.287493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531726c6-5f91-49a8-b109-0aded074ed0b admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:08.287947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531726c6-5f91-49a8-b109-0aded074ed0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:08.288319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531726c6-5f91-49a8-b109-0aded074ed0b admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:08.289258 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531726c6-5f91-49a8-b109-0aded074ed0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:08.291570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531726c6-5f91-49a8-b109-0aded074ed0b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:08.303109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1137/2286] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:08.307616 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec2eae-c6f1-4312-8dd5-e52e871ac919 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:08.318313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec2eae-c6f1-4312-8dd5-e52e871ac919 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:08.323898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1d861-cd56-4777-b2ac-8bbe506c1418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:08.324574 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1d861-cd56-4777-b2ac-8bbe506c1418 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:08.324854 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1d861-cd56-4777-b2ac-8bbe506c1418 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:08.325122 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1d861-cd56-4777-b2ac-8bbe506c1418 admin admin] PATH_INFO: `/v3/projects/a454348c912a40b3b0342137d02793e2/users/e0271ae1e5b84ed78af71dce187bb692/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:08.327283 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1d861-cd56-4777-b2ac-8bbe506c1418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a454348c912a40b3b0342137d02793e2, user_id=e0271ae1e5b84ed78af71dce187bb692, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:08.329912 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1d861-cd56-4777-b2ac-8bbe506c1418 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:08.341086 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1150/2287] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:08 2022] PUT /identity/v3/projects/a454348c912a40b3b0342137d02793e2/users/e0271ae1e5b84ed78af71dce187bb692/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:00:08.351566 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709fe29-4b81-453a-ab1b-19e0cd74a47c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:08.351849 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709fe29-4b81-453a-ab1b-19e0cd74a47c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:08.352151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709fe29-4b81-453a-ab1b-19e0cd74a47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:08.383141 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5709fe29-4b81-453a-ab1b-19e0cd74a47c None None] MFA Rules not processed for user `e0271ae1e5b84ed78af71dce187bb692`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:08.384389 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5709fe29-4b81-453a-ab1b-19e0cd74a47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:08.393882 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:08.462206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1138/2288] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:08.473533 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1d861-cd56-4777-b2ac-8bbe506c1418 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:08.475267 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-83d1d861-cd56-4777-b2ac-8bbe506c1418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:08.482740 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:08.534890 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1d861-cd56-4777-b2ac-8bbe506c1418 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:08.539043 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc688fe-b037-4505-a3ac-911702ca2be2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:08.539651 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc688fe-b037-4505-a3ac-911702ca2be2 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:08.539952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc688fe-b037-4505-a3ac-911702ca2be2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:08.540220 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc688fe-b037-4505-a3ac-911702ca2be2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:08.549580 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc688fe-b037-4505-a3ac-911702ca2be2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:08.550999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc688fe-b037-4505-a3ac-911702ca2be2 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:08.560318 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1151/2289] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:08.642937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709fe29-4b81-453a-ab1b-19e0cd74a47c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:08.707779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709fe29-4b81-453a-ab1b-19e0cd74a47c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:08.713297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73110a60-c041-4842-99f1-d3c5433c5001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:08.714060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73110a60-c041-4842-99f1-d3c5433c5001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:08.714332 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73110a60-c041-4842-99f1-d3c5433c5001 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:08.714609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73110a60-c041-4842-99f1-d3c5433c5001 admin admin] PATH_INFO: `/v3/users/9db9b9d087674632a22beaf010b84688` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:08.715862 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73110a60-c041-4842-99f1-d3c5433c5001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9db9b9d087674632a22beaf010b84688)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:08.717844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73110a60-c041-4842-99f1-d3c5433c5001 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:08.772510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73110a60-c041-4842-99f1-d3c5433c5001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9db9b9d087674632a22beaf010b84688'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:08.773289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73110a60-c041-4842-99f1-d3c5433c5001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9db9b9d087674632a22beaf010b84688'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:08.774028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73110a60-c041-4842-99f1-d3c5433c5001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9db9b9d087674632a22beaf010b84688'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:08.786525 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73110a60-c041-4842-99f1-d3c5433c5001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9db9b9d087674632a22beaf010b84688'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:08.796036 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73110a60-c041-4842-99f1-d3c5433c5001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9db9b9d087674632a22beaf010b84688'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:08.809818 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1139/2290] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:08 2022] DELETE /identity/v3/users/9db9b9d087674632a22beaf010b84688 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:08.814477 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56fc9e-42a5-47fb-aa24-6759e1757090 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:08.814840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56fc9e-42a5-47fb-aa24-6759e1757090 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:08.815155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56fc9e-42a5-47fb-aa24-6759e1757090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:08.834628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-4a56fc9e-42a5-47fb-aa24-6759e1757090 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:08.835830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4a56fc9e-42a5-47fb-aa24-6759e1757090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:08.900557 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1152/2291] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 14:00:08.910876 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73110a60-c041-4842-99f1-d3c5433c5001 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:08.912474 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-73110a60-c041-4842-99f1-d3c5433c5001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:08.940978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:08.974096 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73110a60-c041-4842-99f1-d3c5433c5001 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:08.978570 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75b507-c596-4e91-b822-0cc2915fad63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:08.979189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75b507-c596-4e91-b822-0cc2915fad63 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:08.979924 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75b507-c596-4e91-b822-0cc2915fad63 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:08.980199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75b507-c596-4e91-b822-0cc2915fad63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:08.990776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75b507-c596-4e91-b822-0cc2915fad63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:08.992301 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75b507-c596-4e91-b822-0cc2915fad63 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:09.003276 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1140/2292] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:09.291359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56fc9e-42a5-47fb-aa24-6759e1757090 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:09.293330 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4a56fc9e-42a5-47fb-aa24-6759e1757090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:09.306232 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56fc9e-42a5-47fb-aa24-6759e1757090 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:09.312429 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:09.313095 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:09.313404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:09.313759 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] PATH_INFO: `/v3/projects/fa5684fd117b4e06979f0c2afbe5006c` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:09.315412 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa5684fd117b4e06979f0c2afbe5006c)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:09.321689 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:09.378306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa5684fd117b4e06979f0c2afbe5006c', 'request_id': 'req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:09.387909 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa5684fd117b4e06979f0c2afbe5006c', 'request_id': 'req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:09.394499 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa5684fd117b4e06979f0c2afbe5006c', 'request_id': 'req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:09.405375 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] The token cache is being invalidate because project fa5684fd117b4e06979f0c2afbe5006c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:09.405665 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:09.406333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:09.407851 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1153/2293] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:09 2022] DELETE /identity/v3/projects/fa5684fd117b4e06979f0c2afbe5006c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:09.411973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75b507-c596-4e91-b822-0cc2915fad63 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:09.422647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75b507-c596-4e91-b822-0cc2915fad63 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:09.486806 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5e611-b042-461f-90ad-9f29d20f50c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:09.487453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5e611-b042-461f-90ad-9f29d20f50c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:09.487730 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5e611-b042-461f-90ad-9f29d20f50c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:09.488082 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5e611-b042-461f-90ad-9f29d20f50c4 admin admin] PATH_INFO: `/v3/users/e0271ae1e5b84ed78af71dce187bb692` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:09.489279 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5e611-b042-461f-90ad-9f29d20f50c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0271ae1e5b84ed78af71dce187bb692)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:09.491166 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5e611-b042-461f-90ad-9f29d20f50c4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:09.542336 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-49b5e611-b042-461f-90ad-9f29d20f50c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0271ae1e5b84ed78af71dce187bb692'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:09.542914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-49b5e611-b042-461f-90ad-9f29d20f50c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0271ae1e5b84ed78af71dce187bb692'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:09.543565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-49b5e611-b042-461f-90ad-9f29d20f50c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0271ae1e5b84ed78af71dce187bb692'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:09.555343 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-49b5e611-b042-461f-90ad-9f29d20f50c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0271ae1e5b84ed78af71dce187bb692'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:09.565969 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-49b5e611-b042-461f-90ad-9f29d20f50c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0271ae1e5b84ed78af71dce187bb692'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:09.578676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1141/2294] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:09 2022] DELETE /identity/v3/users/e0271ae1e5b84ed78af71dce187bb692 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:00:09.882235 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:09.883969 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:09.905284 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecccb4-a66b-4584-bdf7-ae5bc49d5411 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:09.955818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:10.000425 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d54ce7-746c-40b4-aabe-ed293899ff6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:10.001250 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d54ce7-746c-40b4-aabe-ed293899ff6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:10.001847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d54ce7-746c-40b4-aabe-ed293899ff6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:10.002067 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d54ce7-746c-40b4-aabe-ed293899ff6b admin admin] PATH_INFO: `/v3/projects/a454348c912a40b3b0342137d02793e2` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:10.003406 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d54ce7-746c-40b4-aabe-ed293899ff6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a454348c912a40b3b0342137d02793e2)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:10.006530 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d54ce7-746c-40b4-aabe-ed293899ff6b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:10.060430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-93d54ce7-746c-40b4-aabe-ed293899ff6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a454348c912a40b3b0342137d02793e2', 'request_id': 'req-93d54ce7-746c-40b4-aabe-ed293899ff6b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:10.070008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-93d54ce7-746c-40b4-aabe-ed293899ff6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a454348c912a40b3b0342137d02793e2', 'request_id': 'req-93d54ce7-746c-40b4-aabe-ed293899ff6b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:10.077390 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-93d54ce7-746c-40b4-aabe-ed293899ff6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a454348c912a40b3b0342137d02793e2', 'request_id': 'req-93d54ce7-746c-40b4-aabe-ed293899ff6b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:10.089482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-93d54ce7-746c-40b4-aabe-ed293899ff6b admin admin] The token cache is being invalidate because project a454348c912a40b3b0342137d02793e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:10.089804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-93d54ce7-746c-40b4-aabe-ed293899ff6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:10.090504 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-93d54ce7-746c-40b4-aabe-ed293899ff6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:10.092259 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1154/2295] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:09 2022] DELETE /identity/v3/projects/a454348c912a40b3b0342137d02793e2 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:10.097781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d67b58-e18a-4ae7-bebd-9ae1af978134 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:10.098078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d67b58-e18a-4ae7-bebd-9ae1af978134 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:10.098342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d67b58-e18a-4ae7-bebd-9ae1af978134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:10.119167 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b1d67b58-e18a-4ae7-bebd-9ae1af978134 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:10.120693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b1d67b58-e18a-4ae7-bebd-9ae1af978134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:10.171817 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:10.193902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1142/2296] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 14:00:10.204018 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabeabe-7591-45cd-9bf5-fca24839bb3e None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:10.204320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabeabe-7591-45cd-9bf5-fca24839bb3e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:10.204588 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabeabe-7591-45cd-9bf5-fca24839bb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:10.226527 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-7cabeabe-7591-45cd-9bf5-fca24839bb3e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:10.227735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-7cabeabe-7591-45cd-9bf5-fca24839bb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:10.236693 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1155/2297] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:10.241359 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d67b58-e18a-4ae7-bebd-9ae1af978134 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:10.251779 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d67b58-e18a-4ae7-bebd-9ae1af978134 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:10.257255 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b5e5e-48b4-4792-895b-9c9c1062bfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:10.257942 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b5e5e-48b4-4792-895b-9c9c1062bfee admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:10.258216 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b5e5e-48b4-4792-895b-9c9c1062bfee admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:10.258505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b5e5e-48b4-4792-895b-9c9c1062bfee admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:10.259216 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b5e5e-48b4-4792-895b-9c9c1062bfee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:00:10.259514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b5e5e-48b4-4792-895b-9c9c1062bfee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:10.261259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b5e5e-48b4-4792-895b-9c9c1062bfee admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:10.282028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1143/2298] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 14:00:10.286405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabeabe-7591-45cd-9bf5-fca24839bb3e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:10.296773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabeabe-7591-45cd-9bf5-fca24839bb3e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:10.302006 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ad550-14d8-43e9-ba00-4f873a9e4ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:10.302607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ad550-14d8-43e9-ba00-4f873a9e4ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:10.302888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ad550-14d8-43e9-ba00-4f873a9e4ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:10.303161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ad550-14d8-43e9-ba00-4f873a9e4ff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:10.303793 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ad550-14d8-43e9-ba00-4f873a9e4ff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:10.305844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ad550-14d8-43e9-ba00-4f873a9e4ff0 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:10.347578 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1156/2299] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:10 2022] POST /identity/v3/projects => generated 393 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:10.352155 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b5e5e-48b4-4792-895b-9c9c1062bfee admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:10.363564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b5e5e-48b4-4792-895b-9c9c1062bfee admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:10.424605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8c324-dc7c-4049-99c7-441fe100c992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:10.425336 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8c324-dc7c-4049-99c7-441fe100c992 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:10.425467 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8c324-dc7c-4049-99c7-441fe100c992 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:10.425661 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8c324-dc7c-4049-99c7-441fe100c992 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:10.426321 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8c324-dc7c-4049-99c7-441fe100c992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:10.428293 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8c324-dc7c-4049-99c7-441fe100c992 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:10.454607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1144/2300] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:10 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:10.457733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ad550-14d8-43e9-ba00-4f873a9e4ff0 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:10.468329 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ad550-14d8-43e9-ba00-4f873a9e4ff0 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:10.473313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96115192-491b-47f2-9050-3158736a57bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:10.473907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96115192-491b-47f2-9050-3158736a57bf admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:10.474183 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96115192-491b-47f2-9050-3158736a57bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:10.474452 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96115192-491b-47f2-9050-3158736a57bf admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:10.475094 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96115192-491b-47f2-9050-3158736a57bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:10.476744 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96115192-491b-47f2-9050-3158736a57bf admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:10.487372 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1157/2301] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:10.491675 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8c324-dc7c-4049-99c7-441fe100c992 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:10.502451 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8c324-dc7c-4049-99c7-441fe100c992 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:10.507938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd01aca-117d-4c15-ad27-ed26af5a6f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:10.508597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd01aca-117d-4c15-ad27-ed26af5a6f8a admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:10.508870 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd01aca-117d-4c15-ad27-ed26af5a6f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:10.509135 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd01aca-117d-4c15-ad27-ed26af5a6f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:10.509770 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd01aca-117d-4c15-ad27-ed26af5a6f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:10.511476 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd01aca-117d-4c15-ad27-ed26af5a6f8a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:10.522416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1145/2302] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:10.526773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96115192-491b-47f2-9050-3158736a57bf admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:10.539009 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96115192-491b-47f2-9050-3158736a57bf admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:10.545502 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f61af6-2a06-40a6-a62d-6f577276caa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:10.546171 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f61af6-2a06-40a6-a62d-6f577276caa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:10.546451 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f61af6-2a06-40a6-a62d-6f577276caa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:10.546726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f61af6-2a06-40a6-a62d-6f577276caa8 admin admin] PATH_INFO: `/v3/projects/0330fde8c21341ad95008a4386ac5080/users/31c3b0fc07d34cfe94bda83f5993daa7/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:10.562085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f61af6-2a06-40a6-a62d-6f577276caa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0330fde8c21341ad95008a4386ac5080, user_id=31c3b0fc07d34cfe94bda83f5993daa7, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:10.564787 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f61af6-2a06-40a6-a62d-6f577276caa8 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:10.580214 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1158/2303] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:10 2022] PUT /identity/v3/projects/0330fde8c21341ad95008a4386ac5080/users/31c3b0fc07d34cfe94bda83f5993daa7/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:10.590583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5db9a-f656-47e9-b6f5-122830bdecc7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:10.590864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5db9a-f656-47e9-b6f5-122830bdecc7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:10.591163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5db9a-f656-47e9-b6f5-122830bdecc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:10.626085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-43f5db9a-f656-47e9-b6f5-122830bdecc7 None None] MFA Rules not processed for user `31c3b0fc07d34cfe94bda83f5993daa7`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:10.627746 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-43f5db9a-f656-47e9-b6f5-122830bdecc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:10.696204 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1146/2304] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:10.701364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f61af6-2a06-40a6-a62d-6f577276caa8 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:10.769756 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:10.773629 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f61af6-2a06-40a6-a62d-6f577276caa8 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:10.779922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5c900-2819-4ad9-9429-77ad29b852e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:10.780567 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5c900-2819-4ad9-9429-77ad29b852e3 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:10.780859 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5c900-2819-4ad9-9429-77ad29b852e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:10.781114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5c900-2819-4ad9-9429-77ad29b852e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:10.781735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5c900-2819-4ad9-9429-77ad29b852e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:10.784198 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5c900-2819-4ad9-9429-77ad29b852e3 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:10.817088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1159/2305] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:10 2022] POST /identity/v3/projects => generated 393 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:10.821471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5db9a-f656-47e9-b6f5-122830bdecc7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:10.833347 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5db9a-f656-47e9-b6f5-122830bdecc7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:10.878002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:10.900014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502ce16-3e07-4ac6-8407-5e748a0bc61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:10.900634 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502ce16-3e07-4ac6-8407-5e748a0bc61c admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:10.900911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502ce16-3e07-4ac6-8407-5e748a0bc61c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:10.901179 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502ce16-3e07-4ac6-8407-5e748a0bc61c admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:10.902359 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502ce16-3e07-4ac6-8407-5e748a0bc61c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:10.904338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502ce16-3e07-4ac6-8407-5e748a0bc61c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:10.931481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1147/2306] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:10 2022] POST /identity/v3/users => generated 303 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 14:00:10.933253 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5c900-2819-4ad9-9429-77ad29b852e3 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:10.944090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5c900-2819-4ad9-9429-77ad29b852e3 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:10.949516 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d2150-4873-4440-a0cf-a7543aa46c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:10.950072 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d2150-4873-4440-a0cf-a7543aa46c42 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:10.950353 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d2150-4873-4440-a0cf-a7543aa46c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:10.950627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d2150-4873-4440-a0cf-a7543aa46c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:10.951280 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d2150-4873-4440-a0cf-a7543aa46c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:10.952941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d2150-4873-4440-a0cf-a7543aa46c42 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:10.964854 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1160/2307] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 14:00:10.971840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502ce16-3e07-4ac6-8407-5e748a0bc61c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:10.982507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502ce16-3e07-4ac6-8407-5e748a0bc61c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:10.989214 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23846d-3014-440e-967f-c70f1cbb342e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:10.989982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23846d-3014-440e-967f-c70f1cbb342e admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:10.990260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23846d-3014-440e-967f-c70f1cbb342e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:10.990558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23846d-3014-440e-967f-c70f1cbb342e admin admin] PATH_INFO: `/v3/projects/ac269cd714724d6f9d82cb85f748ba79/users/8cd0823776bd4168823236b119a18b34/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:11.007532 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23846d-3014-440e-967f-c70f1cbb342e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac269cd714724d6f9d82cb85f748ba79, user_id=8cd0823776bd4168823236b119a18b34, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:11.010314 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23846d-3014-440e-967f-c70f1cbb342e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:11.026567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1148/2308] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:10 2022] PUT /identity/v3/projects/ac269cd714724d6f9d82cb85f748ba79/users/8cd0823776bd4168823236b119a18b34/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:11.033363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d2150-4873-4440-a0cf-a7543aa46c42 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:11.041353 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d2150-4873-4440-a0cf-a7543aa46c42 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:11.099931 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765d5fd-7c5c-46f7-970f-35b2d5e0fe42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:11.100617 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765d5fd-7c5c-46f7-970f-35b2d5e0fe42 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:11.100895 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765d5fd-7c5c-46f7-970f-35b2d5e0fe42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:11.101168 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765d5fd-7c5c-46f7-970f-35b2d5e0fe42 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:11.101808 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765d5fd-7c5c-46f7-970f-35b2d5e0fe42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:11.103502 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765d5fd-7c5c-46f7-970f-35b2d5e0fe42 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:11.117991 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1161/2309] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:11.122277 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23846d-3014-440e-967f-c70f1cbb342e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:11.133400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23846d-3014-440e-967f-c70f1cbb342e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:11.138719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a4a7c-cb0d-4d5c-a801-9412a9301f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:11.139398 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a4a7c-cb0d-4d5c-a801-9412a9301f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:11.139676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a4a7c-cb0d-4d5c-a801-9412a9301f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:11.139983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a4a7c-cb0d-4d5c-a801-9412a9301f4e admin admin] PATH_INFO: `/v3/projects/ac269cd714724d6f9d82cb85f748ba79/users/8cd0823776bd4168823236b119a18b34/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:11.142041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a4a7c-cb0d-4d5c-a801-9412a9301f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac269cd714724d6f9d82cb85f748ba79, user_id=8cd0823776bd4168823236b119a18b34, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:11.144730 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a4a7c-cb0d-4d5c-a801-9412a9301f4e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:11.155784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1149/2310] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:11 2022] PUT /identity/v3/projects/ac269cd714724d6f9d82cb85f748ba79/users/8cd0823776bd4168823236b119a18b34/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:11.166135 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec4589-b213-4e1a-860d-09b9a59610b3 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:11.166420 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec4589-b213-4e1a-860d-09b9a59610b3 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:11.166683 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec4589-b213-4e1a-860d-09b9a59610b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:11.197656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-9dec4589-b213-4e1a-860d-09b9a59610b3 None None] MFA Rules not processed for user `8cd0823776bd4168823236b119a18b34`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:11.198888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9dec4589-b213-4e1a-860d-09b9a59610b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:11.210320 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a4a7c-cb0d-4d5c-a801-9412a9301f4e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:11.212387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5f4a4a7c-cb0d-4d5c-a801-9412a9301f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:11.269794 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1162/2311] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:11 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:11.278105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a4a7c-cb0d-4d5c-a801-9412a9301f4e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:11.281039 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec4589-b213-4e1a-860d-09b9a59610b3 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:11.282668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9dec4589-b213-4e1a-860d-09b9a59610b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:11.283368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43667116-2f4b-48c1-a141-ab37b3d46239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:11.284117 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43667116-2f4b-48c1-a141-ab37b3d46239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:11.284388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43667116-2f4b-48c1-a141-ab37b3d46239 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:11.284657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43667116-2f4b-48c1-a141-ab37b3d46239 admin admin] PATH_INFO: `/v3/users/923311fd047b41eaafded69d348a95c3` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:11.285789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43667116-2f4b-48c1-a141-ab37b3d46239 admin admin] RBAC: Authorizing `identity:delete_user(user_id=923311fd047b41eaafded69d348a95c3)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:11.287745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43667116-2f4b-48c1-a141-ab37b3d46239 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:11.340402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-43667116-2f4b-48c1-a141-ab37b3d46239 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '923311fd047b41eaafded69d348a95c3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:11.341294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-43667116-2f4b-48c1-a141-ab37b3d46239 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '923311fd047b41eaafded69d348a95c3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:11.341923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-43667116-2f4b-48c1-a141-ab37b3d46239 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '923311fd047b41eaafded69d348a95c3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:11.343279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec4589-b213-4e1a-860d-09b9a59610b3 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:11.351326 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887135c4-cf75-40de-87ba-0d9cff0fe080 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:11.352248 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887135c4-cf75-40de-87ba-0d9cff0fe080 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:11.352593 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887135c4-cf75-40de-87ba-0d9cff0fe080 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:11.352923 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887135c4-cf75-40de-87ba-0d9cff0fe080 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:11.355530 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-887135c4-cf75-40de-87ba-0d9cff0fe080 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:11.355984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-43667116-2f4b-48c1-a141-ab37b3d46239 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '923311fd047b41eaafded69d348a95c3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:11.365769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-43667116-2f4b-48c1-a141-ab37b3d46239 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '923311fd047b41eaafded69d348a95c3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:11.367790 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:11.370939 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887135c4-cf75-40de-87ba-0d9cff0fe080 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:11.372136 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887135c4-cf75-40de-87ba-0d9cff0fe080 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:11.384986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1150/2312] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:11 2022] DELETE /identity/v3/users/923311fd047b41eaafded69d348a95c3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:11.385288 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1163/2313] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:11 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:11.389672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40caa277-cfcd-43a8-b56e-d0a35ce9978d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:11.390007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40caa277-cfcd-43a8-b56e-d0a35ce9978d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:11.390291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40caa277-cfcd-43a8-b56e-d0a35ce9978d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:11.416299 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-40caa277-cfcd-43a8-b56e-d0a35ce9978d None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:11.417602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-40caa277-cfcd-43a8-b56e-d0a35ce9978d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:11.426283 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:11.449708 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887135c4-cf75-40de-87ba-0d9cff0fe080 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:11.451220 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-887135c4-cf75-40de-87ba-0d9cff0fe080 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:11.462227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887135c4-cf75-40de-87ba-0d9cff0fe080 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:11.483416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1151/2314] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:11.493579 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40caa277-cfcd-43a8-b56e-d0a35ce9978d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:11.495108 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-40caa277-cfcd-43a8-b56e-d0a35ce9978d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:11.514849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59dfda-3e29-4d04-b738-0804bcbaa046 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:11.515468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59dfda-3e29-4d04-b738-0804bcbaa046 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:11.515750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59dfda-3e29-4d04-b738-0804bcbaa046 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:11.516066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59dfda-3e29-4d04-b738-0804bcbaa046 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:11.517893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9b59dfda-3e29-4d04-b738-0804bcbaa046 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:11.554516 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40caa277-cfcd-43a8-b56e-d0a35ce9978d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:11.558886 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a77f6-b370-4262-a385-b7aca6b50f1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:11.559643 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a77f6-b370-4262-a385-b7aca6b50f1e service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:11.559958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a77f6-b370-4262-a385-b7aca6b50f1e service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:11.560226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a77f6-b370-4262-a385-b7aca6b50f1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:11.569976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a77f6-b370-4262-a385-b7aca6b50f1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:11.571441 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a77f6-b370-4262-a385-b7aca6b50f1e service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:11.585659 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1152/2315] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 14:00:11.588030 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b59dfda-3e29-4d04-b738-0804bcbaa046 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:11.589490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b59dfda-3e29-4d04-b738-0804bcbaa046 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:11.599358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1164/2316] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 14:00:11.879078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a77f6-b370-4262-a385-b7aca6b50f1e service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:11.882243 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e51a77f6-b370-4262-a385-b7aca6b50f1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:11.908104 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a77f6-b370-4262-a385-b7aca6b50f1e service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:11.913159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:11.913796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:11.914072 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:11.914342 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] PATH_INFO: `/v3/projects/0b57c243763e4416be29f33b5e76dfec` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:11.915514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b57c243763e4416be29f33b5e76dfec)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:11.918152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:11.983113 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b57c243763e4416be29f33b5e76dfec', 'request_id': 'req-bbb8090f-6824-466c-9574-8fd5705d7cc6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:11.992819 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b57c243763e4416be29f33b5e76dfec', 'request_id': 'req-bbb8090f-6824-466c-9574-8fd5705d7cc6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:12.004771 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b57c243763e4416be29f33b5e76dfec', 'request_id': 'req-bbb8090f-6824-466c-9574-8fd5705d7cc6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:12.017717 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] The token cache is being invalidate because project 0b57c243763e4416be29f33b5e76dfec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:12.018000 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:12.018644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:12.020213 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1153/2317] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:11 2022] DELETE /identity/v3/projects/0b57c243763e4416be29f33b5e76dfec => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:12.024286 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59dfda-3e29-4d04-b738-0804bcbaa046 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:12.035671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59dfda-3e29-4d04-b738-0804bcbaa046 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:12.104844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b155f3-9e26-42ed-8999-2ebba896c6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:12.105473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b155f3-9e26-42ed-8999-2ebba896c6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:12.105747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b155f3-9e26-42ed-8999-2ebba896c6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:12.106021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b155f3-9e26-42ed-8999-2ebba896c6f5 admin admin] PATH_INFO: `/v3/users/0b5226b387ef42b09ab2c651d780fc6f` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:12.107165 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b155f3-9e26-42ed-8999-2ebba896c6f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b5226b387ef42b09ab2c651d780fc6f)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:12.109201 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b155f3-9e26-42ed-8999-2ebba896c6f5 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:12.165156 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-61b155f3-9e26-42ed-8999-2ebba896c6f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b5226b387ef42b09ab2c651d780fc6f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:12.169880 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:12.173774 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-61b155f3-9e26-42ed-8999-2ebba896c6f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b5226b387ef42b09ab2c651d780fc6f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:12.174862 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-61b155f3-9e26-42ed-8999-2ebba896c6f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b5226b387ef42b09ab2c651d780fc6f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:12.175521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-61b155f3-9e26-42ed-8999-2ebba896c6f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b5226b387ef42b09ab2c651d780fc6f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:12.185871 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-61b155f3-9e26-42ed-8999-2ebba896c6f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b5226b387ef42b09ab2c651d780fc6f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:12.203713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1165/2318] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:12 2022] DELETE /identity/v3/users/0b5226b387ef42b09ab2c651d780fc6f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:12.496098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:12.497824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:12.513568 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8090f-6824-466c-9574-8fd5705d7cc6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:12.553594 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:12.577796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63775bab-fe48-47b2-9aa9-969e2817dc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:12.578361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63775bab-fe48-47b2-9aa9-969e2817dc70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:12.578635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63775bab-fe48-47b2-9aa9-969e2817dc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:12.578930 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63775bab-fe48-47b2-9aa9-969e2817dc70 admin admin] PATH_INFO: `/v3/projects/ea924dde650b4cfc9a26878c0f01c302` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:12.580177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63775bab-fe48-47b2-9aa9-969e2817dc70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea924dde650b4cfc9a26878c0f01c302)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:12.582101 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63775bab-fe48-47b2-9aa9-969e2817dc70 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:12.633792 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-63775bab-fe48-47b2-9aa9-969e2817dc70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea924dde650b4cfc9a26878c0f01c302', 'request_id': 'req-63775bab-fe48-47b2-9aa9-969e2817dc70'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:12.641996 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-63775bab-fe48-47b2-9aa9-969e2817dc70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea924dde650b4cfc9a26878c0f01c302', 'request_id': 'req-63775bab-fe48-47b2-9aa9-969e2817dc70'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:12.651311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-63775bab-fe48-47b2-9aa9-969e2817dc70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea924dde650b4cfc9a26878c0f01c302', 'request_id': 'req-63775bab-fe48-47b2-9aa9-969e2817dc70'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:12.666303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-63775bab-fe48-47b2-9aa9-969e2817dc70 admin admin] The token cache is being invalidate because project ea924dde650b4cfc9a26878c0f01c302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:12.666668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-63775bab-fe48-47b2-9aa9-969e2817dc70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:12.667747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-63775bab-fe48-47b2-9aa9-969e2817dc70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:12.669988 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1154/2319] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:12 2022] DELETE /identity/v3/projects/ea924dde650b4cfc9a26878c0f01c302 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:12.675171 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3451b3f-b3de-4fd7-9892-96483b0fca12 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:12.675646 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3451b3f-b3de-4fd7-9892-96483b0fca12 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:12.675817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3451b3f-b3de-4fd7-9892-96483b0fca12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:12.696027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-c3451b3f-b3de-4fd7-9892-96483b0fca12 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:12.697242 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c3451b3f-b3de-4fd7-9892-96483b0fca12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:12.758386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1166/2320] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:12.767408 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789e0a1-c158-46be-98f9-13c13f241540 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:12.767692 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789e0a1-c158-46be-98f9-13c13f241540 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:12.767989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789e0a1-c158-46be-98f9-13c13f241540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:12.789694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d789e0a1-c158-46be-98f9-13c13f241540 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:12.790861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d789e0a1-c158-46be-98f9-13c13f241540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:12.800618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1155/2321] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:12.805202 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3451b3f-b3de-4fd7-9892-96483b0fca12 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:12.815501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3451b3f-b3de-4fd7-9892-96483b0fca12 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:12.820350 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa6b67-f47c-46cb-ab0b-a6b8eb69ab08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:12.820960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa6b67-f47c-46cb-ab0b-a6b8eb69ab08 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:12.821241 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa6b67-f47c-46cb-ab0b-a6b8eb69ab08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:12.821513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa6b67-f47c-46cb-ab0b-a6b8eb69ab08 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:12.822182 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa6b67-f47c-46cb-ab0b-a6b8eb69ab08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:00:12.822465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa6b67-f47c-46cb-ab0b-a6b8eb69ab08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:12.824261 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa6b67-f47c-46cb-ab0b-a6b8eb69ab08 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:12.836148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1167/2322] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 14:00:12.840675 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789e0a1-c158-46be-98f9-13c13f241540 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:12.850881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789e0a1-c158-46be-98f9-13c13f241540 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:12.857384 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb3b70-e53e-4e96-935d-8cdcdfa8bfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:12.858015 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb3b70-e53e-4e96-935d-8cdcdfa8bfa6 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:12.858257 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb3b70-e53e-4e96-935d-8cdcdfa8bfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:12.858518 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb3b70-e53e-4e96-935d-8cdcdfa8bfa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:12.859138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb3b70-e53e-4e96-935d-8cdcdfa8bfa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:12.861084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb3b70-e53e-4e96-935d-8cdcdfa8bfa6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:12.892268 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1156/2323] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:12 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:12.898518 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa6b67-f47c-46cb-ab0b-a6b8eb69ab08 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:12.913077 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa6b67-f47c-46cb-ab0b-a6b8eb69ab08 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:12.972050 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88929e6-f6e3-482e-9979-a160113c6487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:12.972671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88929e6-f6e3-482e-9979-a160113c6487 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:12.972949 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88929e6-f6e3-482e-9979-a160113c6487 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:12.973219 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88929e6-f6e3-482e-9979-a160113c6487 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:12.973858 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88929e6-f6e3-482e-9979-a160113c6487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:12.975785 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88929e6-f6e3-482e-9979-a160113c6487 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:13.000821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1168/2324] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:12 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:13.007640 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb3b70-e53e-4e96-935d-8cdcdfa8bfa6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:13.019895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb3b70-e53e-4e96-935d-8cdcdfa8bfa6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:13.025283 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1864615-c153-4081-897e-6c274a0763ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:13.025897 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1864615-c153-4081-897e-6c274a0763ec admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:13.026194 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1864615-c153-4081-897e-6c274a0763ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:13.026609 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1864615-c153-4081-897e-6c274a0763ec admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:13.027380 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1864615-c153-4081-897e-6c274a0763ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:13.029028 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1864615-c153-4081-897e-6c274a0763ec admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:13.039676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1157/2325] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:13.044904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88929e6-f6e3-482e-9979-a160113c6487 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:13.061401 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88929e6-f6e3-482e-9979-a160113c6487 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:13.070174 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e64bf7-2413-476a-80dc-584733a1f1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:13.071021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e64bf7-2413-476a-80dc-584733a1f1ce admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:13.071402 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e64bf7-2413-476a-80dc-584733a1f1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:13.071773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e64bf7-2413-476a-80dc-584733a1f1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:13.072791 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e64bf7-2413-476a-80dc-584733a1f1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:13.074794 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e64bf7-2413-476a-80dc-584733a1f1ce admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:13.085867 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1169/2326] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:13.090586 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1864615-c153-4081-897e-6c274a0763ec admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:13.101214 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1864615-c153-4081-897e-6c274a0763ec admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:13.107103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b994e3e-dae8-4987-b1fc-fc1c15603ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:13.107755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994e3e-dae8-4987-b1fc-fc1c15603ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:13.108141 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994e3e-dae8-4987-b1fc-fc1c15603ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:13.108417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994e3e-dae8-4987-b1fc-fc1c15603ac1 admin admin] PATH_INFO: `/v3/projects/e21b59cec508431ebd44bf7c6bf046d4/users/3879b31711294f3fa2d60d120c2ea9f0/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:13.123514 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b994e3e-dae8-4987-b1fc-fc1c15603ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e21b59cec508431ebd44bf7c6bf046d4, user_id=3879b31711294f3fa2d60d120c2ea9f0, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:13.126231 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b994e3e-dae8-4987-b1fc-fc1c15603ac1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:13.146333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1158/2327] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:13 2022] PUT /identity/v3/projects/e21b59cec508431ebd44bf7c6bf046d4/users/3879b31711294f3fa2d60d120c2ea9f0/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:13.160074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70725927-aec2-43e0-829c-129a595cb6c4 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:13.160356 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70725927-aec2-43e0-829c-129a595cb6c4 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:13.160627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70725927-aec2-43e0-829c-129a595cb6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:13.195388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-70725927-aec2-43e0-829c-129a595cb6c4 None None] MFA Rules not processed for user `3879b31711294f3fa2d60d120c2ea9f0`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:13.196634 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-70725927-aec2-43e0-829c-129a595cb6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:13.222748 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:13.268707 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1170/2328] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:13.274067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b994e3e-dae8-4987-b1fc-fc1c15603ac1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:13.348774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b994e3e-dae8-4987-b1fc-fc1c15603ac1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:13.357954 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efd382-0afd-4404-9ea7-8fbf51215308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:13.358784 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efd382-0afd-4404-9ea7-8fbf51215308 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:13.359139 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efd382-0afd-4404-9ea7-8fbf51215308 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:13.359492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efd382-0afd-4404-9ea7-8fbf51215308 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:13.360388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efd382-0afd-4404-9ea7-8fbf51215308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:13.363427 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efd382-0afd-4404-9ea7-8fbf51215308 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:13.377627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:13.398406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1159/2329] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:13 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:13.404146 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70725927-aec2-43e0-829c-129a595cb6c4 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:13.416132 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70725927-aec2-43e0-829c-129a595cb6c4 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:13.476268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6882c7-a4b4-4194-b31b-ffee856ee887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:13.476880 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6882c7-a4b4-4194-b31b-ffee856ee887 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:13.477154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6882c7-a4b4-4194-b31b-ffee856ee887 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:13.477423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6882c7-a4b4-4194-b31b-ffee856ee887 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:13.478046 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6882c7-a4b4-4194-b31b-ffee856ee887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:13.480369 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6882c7-a4b4-4194-b31b-ffee856ee887 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:13.505054 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1171/2330] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:13 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:13.509650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efd382-0afd-4404-9ea7-8fbf51215308 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:13.521290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efd382-0afd-4404-9ea7-8fbf51215308 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:13.526730 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac9355-167e-498a-b7e3-24babd733bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:13.527360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac9355-167e-498a-b7e3-24babd733bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:13.527633 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac9355-167e-498a-b7e3-24babd733bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:13.527896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac9355-167e-498a-b7e3-24babd733bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:13.528560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac9355-167e-498a-b7e3-24babd733bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:13.530197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac9355-167e-498a-b7e3-24babd733bd4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:13.540415 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1160/2331] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:13.544771 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6882c7-a4b4-4194-b31b-ffee856ee887 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:13.554843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6882c7-a4b4-4194-b31b-ffee856ee887 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:13.559589 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a04fa-52df-4c81-b650-1670357d75eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:13.560269 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a04fa-52df-4c81-b650-1670357d75eb admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:13.560541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a04fa-52df-4c81-b650-1670357d75eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:13.560811 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a04fa-52df-4c81-b650-1670357d75eb admin admin] PATH_INFO: `/v3/projects/ed7522646b9f4b6c98c392c93784d2d1/users/397c81306de44e048d61002335988197/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:13.576251 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a04fa-52df-4c81-b650-1670357d75eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7522646b9f4b6c98c392c93784d2d1, user_id=397c81306de44e048d61002335988197, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:13.578892 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a04fa-52df-4c81-b650-1670357d75eb admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:13.597289 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1172/2332] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:13 2022] PUT /identity/v3/projects/ed7522646b9f4b6c98c392c93784d2d1/users/397c81306de44e048d61002335988197/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:13.604449 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac9355-167e-498a-b7e3-24babd733bd4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:13.617056 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac9355-167e-498a-b7e3-24babd733bd4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:13.687140 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66ca03-fb79-45f5-9e9f-e4a98d621424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:13.688020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66ca03-fb79-45f5-9e9f-e4a98d621424 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:13.688423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66ca03-fb79-45f5-9e9f-e4a98d621424 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:13.688710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66ca03-fb79-45f5-9e9f-e4a98d621424 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:13.689621 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66ca03-fb79-45f5-9e9f-e4a98d621424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:13.692031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66ca03-fb79-45f5-9e9f-e4a98d621424 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:13.704590 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1161/2333] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:13.708970 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a04fa-52df-4c81-b650-1670357d75eb admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:13.719905 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a04fa-52df-4c81-b650-1670357d75eb admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:13.725335 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba9c3e-6c38-48a8-a979-086c72bf41db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:13.725995 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba9c3e-6c38-48a8-a979-086c72bf41db admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:13.726266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba9c3e-6c38-48a8-a979-086c72bf41db admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:13.726537 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba9c3e-6c38-48a8-a979-086c72bf41db admin admin] PATH_INFO: `/v3/projects/ed7522646b9f4b6c98c392c93784d2d1/users/397c81306de44e048d61002335988197/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:13.728611 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba9c3e-6c38-48a8-a979-086c72bf41db admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7522646b9f4b6c98c392c93784d2d1, user_id=397c81306de44e048d61002335988197, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:13.731685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba9c3e-6c38-48a8-a979-086c72bf41db admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:13.742993 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1173/2334] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:13 2022] PUT /identity/v3/projects/ed7522646b9f4b6c98c392c93784d2d1/users/397c81306de44e048d61002335988197/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:13.753684 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521817e6-0d15-4a61-8802-af08d76d8ea2 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:13.753973 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521817e6-0d15-4a61-8802-af08d76d8ea2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:13.754248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521817e6-0d15-4a61-8802-af08d76d8ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:13.783149 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-521817e6-0d15-4a61-8802-af08d76d8ea2 None None] MFA Rules not processed for user `397c81306de44e048d61002335988197`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:13.784397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-521817e6-0d15-4a61-8802-af08d76d8ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:13.869272 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1162/2335] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:13.881216 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba9c3e-6c38-48a8-a979-086c72bf41db admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:13.883355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b6ba9c3e-6c38-48a8-a979-086c72bf41db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:13.944225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba9c3e-6c38-48a8-a979-086c72bf41db admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:13.948161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f019d-e7e3-4c3b-93a2-d251abc5804e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:13.948796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f019d-e7e3-4c3b-93a2-d251abc5804e service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:13.949073 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f019d-e7e3-4c3b-93a2-d251abc5804e service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:13.949343 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f019d-e7e3-4c3b-93a2-d251abc5804e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:13.958781 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f019d-e7e3-4c3b-93a2-d251abc5804e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:13.960260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f019d-e7e3-4c3b-93a2-d251abc5804e service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:13.970021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1174/2336] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:13.999792 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521817e6-0d15-4a61-8802-af08d76d8ea2 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:14.072565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521817e6-0d15-4a61-8802-af08d76d8ea2 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:14.077448 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fe22a-1bef-4af4-9077-3e04a9166420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:14.078095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fe22a-1bef-4af4-9077-3e04a9166420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:14.078381 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fe22a-1bef-4af4-9077-3e04a9166420 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:14.078878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fe22a-1bef-4af4-9077-3e04a9166420 admin admin] PATH_INFO: `/v3/users/3879b31711294f3fa2d60d120c2ea9f0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:14.080063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18fe22a-1bef-4af4-9077-3e04a9166420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3879b31711294f3fa2d60d120c2ea9f0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:14.082010 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18fe22a-1bef-4af4-9077-3e04a9166420 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:14.084569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:14.135100 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a18fe22a-1bef-4af4-9077-3e04a9166420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3879b31711294f3fa2d60d120c2ea9f0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.135817 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a18fe22a-1bef-4af4-9077-3e04a9166420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3879b31711294f3fa2d60d120c2ea9f0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.136529 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a18fe22a-1bef-4af4-9077-3e04a9166420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3879b31711294f3fa2d60d120c2ea9f0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.149423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a18fe22a-1bef-4af4-9077-3e04a9166420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3879b31711294f3fa2d60d120c2ea9f0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.164130 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a18fe22a-1bef-4af4-9077-3e04a9166420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3879b31711294f3fa2d60d120c2ea9f0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.180470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1163/2337] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:13 2022] DELETE /identity/v3/users/3879b31711294f3fa2d60d120c2ea9f0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:14.185324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12509c25-3f21-47cf-92bd-9066d2925c07 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:14.185612 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12509c25-3f21-47cf-92bd-9066d2925c07 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:14.185888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12509c25-3f21-47cf-92bd-9066d2925c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:14.206562 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-12509c25-3f21-47cf-92bd-9066d2925c07 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:14.207814 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-12509c25-3f21-47cf-92bd-9066d2925c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:14.221513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:14.270087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1175/2338] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:14.280349 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fe22a-1bef-4af4-9077-3e04a9166420 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:14.281955 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a18fe22a-1bef-4af4-9077-3e04a9166420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:14.341364 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fe22a-1bef-4af4-9077-3e04a9166420 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:14.346482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531bfd8-686c-4106-afcf-102ca902addd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:14.347088 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531bfd8-686c-4106-afcf-102ca902addd service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:14.347358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531bfd8-686c-4106-afcf-102ca902addd service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:14.347618 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531bfd8-686c-4106-afcf-102ca902addd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:14.360168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531bfd8-686c-4106-afcf-102ca902addd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:14.361613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531bfd8-686c-4106-afcf-102ca902addd service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:14.371370 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1164/2339] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:14.675054 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12509c25-3f21-47cf-92bd-9066d2925c07 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:14.677672 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-12509c25-3f21-47cf-92bd-9066d2925c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:14.698212 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12509c25-3f21-47cf-92bd-9066d2925c07 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:14.706288 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:14.707258 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:14.707755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:14.708174 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] PATH_INFO: `/v3/projects/e21b59cec508431ebd44bf7c6bf046d4` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:14.709430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e21b59cec508431ebd44bf7c6bf046d4)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:14.711821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:14.764836 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e21b59cec508431ebd44bf7c6bf046d4', 'request_id': 'req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.775154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e21b59cec508431ebd44bf7c6bf046d4', 'request_id': 'req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.782432 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e21b59cec508431ebd44bf7c6bf046d4', 'request_id': 'req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.794000 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] The token cache is being invalidate because project e21b59cec508431ebd44bf7c6bf046d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:14.794301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.795024 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.796640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1176/2340] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:14 2022] DELETE /identity/v3/projects/e21b59cec508431ebd44bf7c6bf046d4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:14.801162 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531bfd8-686c-4106-afcf-102ca902addd service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:14.811830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531bfd8-686c-4106-afcf-102ca902addd service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:14.876301 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce59f48-d66a-49b0-bd03-eb5dbc2521a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:14.876952 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce59f48-d66a-49b0-bd03-eb5dbc2521a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:14.877228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce59f48-d66a-49b0-bd03-eb5dbc2521a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:14.877500 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce59f48-d66a-49b0-bd03-eb5dbc2521a1 admin admin] PATH_INFO: `/v3/users/397c81306de44e048d61002335988197` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:14.878704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce59f48-d66a-49b0-bd03-eb5dbc2521a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=397c81306de44e048d61002335988197)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:14.880673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce59f48-d66a-49b0-bd03-eb5dbc2521a1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:14.910787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:14.931471 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4ce59f48-d66a-49b0-bd03-eb5dbc2521a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '397c81306de44e048d61002335988197'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.932107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4ce59f48-d66a-49b0-bd03-eb5dbc2521a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '397c81306de44e048d61002335988197'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.932794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4ce59f48-d66a-49b0-bd03-eb5dbc2521a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '397c81306de44e048d61002335988197'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.944624 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4ce59f48-d66a-49b0-bd03-eb5dbc2521a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '397c81306de44e048d61002335988197'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.954078 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4ce59f48-d66a-49b0-bd03-eb5dbc2521a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '397c81306de44e048d61002335988197'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:14.967319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1165/2341] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:14 2022] DELETE /identity/v3/users/397c81306de44e048d61002335988197 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:15.250938 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:15.252585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:15.267191 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c610c-f590-4be7-bc25-af4e4b2c9c0e admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:15.328489 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44122f2c-8725-44f1-b6e3-bd4a355e6b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:15.329143 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44122f2c-8725-44f1-b6e3-bd4a355e6b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:15.329423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44122f2c-8725-44f1-b6e3-bd4a355e6b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:15.329697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44122f2c-8725-44f1-b6e3-bd4a355e6b59 admin admin] PATH_INFO: `/v3/projects/ed7522646b9f4b6c98c392c93784d2d1` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:15.330842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44122f2c-8725-44f1-b6e3-bd4a355e6b59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed7522646b9f4b6c98c392c93784d2d1)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:15.332800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44122f2c-8725-44f1-b6e3-bd4a355e6b59 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:15.391149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:15.393906 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-44122f2c-8725-44f1-b6e3-bd4a355e6b59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed7522646b9f4b6c98c392c93784d2d1', 'request_id': 'req-44122f2c-8725-44f1-b6e3-bd4a355e6b59'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:15.403260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-44122f2c-8725-44f1-b6e3-bd4a355e6b59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed7522646b9f4b6c98c392c93784d2d1', 'request_id': 'req-44122f2c-8725-44f1-b6e3-bd4a355e6b59'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:15.410419 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-44122f2c-8725-44f1-b6e3-bd4a355e6b59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed7522646b9f4b6c98c392c93784d2d1', 'request_id': 'req-44122f2c-8725-44f1-b6e3-bd4a355e6b59'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:15.421264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-44122f2c-8725-44f1-b6e3-bd4a355e6b59 admin admin] The token cache is being invalidate because project ed7522646b9f4b6c98c392c93784d2d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:15.421556 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-44122f2c-8725-44f1-b6e3-bd4a355e6b59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:15.422220 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-44122f2c-8725-44f1-b6e3-bd4a355e6b59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:15.423746 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1177/2342] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:15 2022] DELETE /identity/v3/projects/ed7522646b9f4b6c98c392c93784d2d1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:00:15.428742 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52872b-eeb8-4948-a6a7-312a794767e7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:15.429095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52872b-eeb8-4948-a6a7-312a794767e7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:15.429434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52872b-eeb8-4948-a6a7-312a794767e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:15.449180 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-cf52872b-eeb8-4948-a6a7-312a794767e7 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:15.450302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cf52872b-eeb8-4948-a6a7-312a794767e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:15.510075 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1166/2343] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 14:00:15.521121 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0b1e0-0b10-4d45-9555-877300f13d75 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:15.521392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0b1e0-0b10-4d45-9555-877300f13d75 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:15.521819 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0b1e0-0b10-4d45-9555-877300f13d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:15.544201 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-35f0b1e0-0b10-4d45-9555-877300f13d75 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:15.545409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-35f0b1e0-0b10-4d45-9555-877300f13d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:15.553889 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1178/2344] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:15.558368 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52872b-eeb8-4948-a6a7-312a794767e7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:15.568307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52872b-eeb8-4948-a6a7-312a794767e7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:15.573177 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3947b-a1ee-4954-a96e-7248d5b763e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:15.573813 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3947b-a1ee-4954-a96e-7248d5b763e1 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:15.574087 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3947b-a1ee-4954-a96e-7248d5b763e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:15.574351 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3947b-a1ee-4954-a96e-7248d5b763e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:15.574994 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3947b-a1ee-4954-a96e-7248d5b763e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:00:15.575291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3947b-a1ee-4954-a96e-7248d5b763e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:15.576976 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3947b-a1ee-4954-a96e-7248d5b763e1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:15.588775 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1167/2345] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 14:00:15.593356 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0b1e0-0b10-4d45-9555-877300f13d75 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:15.603365 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0b1e0-0b10-4d45-9555-877300f13d75 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:15.610999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cfbfa2-2cdc-4f93-b5fc-99da693aefd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:15.611628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfbfa2-2cdc-4f93-b5fc-99da693aefd8 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:15.611934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfbfa2-2cdc-4f93-b5fc-99da693aefd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:15.612208 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cfbfa2-2cdc-4f93-b5fc-99da693aefd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:15.612864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cfbfa2-2cdc-4f93-b5fc-99da693aefd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:15.614799 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cfbfa2-2cdc-4f93-b5fc-99da693aefd8 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:15.647735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1179/2346] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:15 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:15.648955 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3947b-a1ee-4954-a96e-7248d5b763e1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:15.661224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3947b-a1ee-4954-a96e-7248d5b763e1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:15.717742 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5ffee-1e5d-4e2e-b57d-87cb193f2fd9 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:15.718309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5ffee-1e5d-4e2e-b57d-87cb193f2fd9 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:15.718309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5ffee-1e5d-4e2e-b57d-87cb193f2fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:15.719962 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca21a2-20b6-43be-bc2c-c0896d5ad0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:15.720593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca21a2-20b6-43be-bc2c-c0896d5ad0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:15.720861 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca21a2-20b6-43be-bc2c-c0896d5ad0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:15.721266 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca21a2-20b6-43be-bc2c-c0896d5ad0b5 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:15.721904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca21a2-20b6-43be-bc2c-c0896d5ad0b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:15.723814 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca21a2-20b6-43be-bc2c-c0896d5ad0b5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:15.740019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-0cf5ffee-1e5d-4e2e-b57d-87cb193f2fd9 None None] MFA Rules not processed for user `193bf7a17f544770a27c07f6b8cc0988`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:15.741346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0cf5ffee-1e5d-4e2e-b57d-87cb193f2fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:15.746638 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1168/2347] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:15 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:15.750934 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca21a2-20b6-43be-bc2c-c0896d5ad0b5 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:15.760800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca21a2-20b6-43be-bc2c-c0896d5ad0b5 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:15.766498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92573b11-ff62-40eb-b404-8296ab04bf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:15.767749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92573b11-ff62-40eb-b404-8296ab04bf8b admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:15.768065 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92573b11-ff62-40eb-b404-8296ab04bf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:15.768340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92573b11-ff62-40eb-b404-8296ab04bf8b admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:15.769387 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92573b11-ff62-40eb-b404-8296ab04bf8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:15.771006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92573b11-ff62-40eb-b404-8296ab04bf8b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:15.782063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1169/2348] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 14:00:15.786095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92573b11-ff62-40eb-b404-8296ab04bf8b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:15.796397 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92573b11-ff62-40eb-b404-8296ab04bf8b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:15.799781 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1180/2349] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:15.801340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe58d7a-9cbc-428e-813c-94cf1083c616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:15.802084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe58d7a-9cbc-428e-813c-94cf1083c616 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:15.802246 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe58d7a-9cbc-428e-813c-94cf1083c616 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:15.802479 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe58d7a-9cbc-428e-813c-94cf1083c616 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:15.803125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe58d7a-9cbc-428e-813c-94cf1083c616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:15.804778 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe58d7a-9cbc-428e-813c-94cf1083c616 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:15.807322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:15.817103 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1170/2350] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 14:00:15.823938 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5ffee-1e5d-4e2e-b57d-87cb193f2fd9 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:15.835368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5ffee-1e5d-4e2e-b57d-87cb193f2fd9 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:15.840489 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d19bb2-8fc8-4086-af6c-a22cee49c451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:15.841163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d19bb2-8fc8-4086-af6c-a22cee49c451 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:15.841496 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d19bb2-8fc8-4086-af6c-a22cee49c451 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:15.841770 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d19bb2-8fc8-4086-af6c-a22cee49c451 admin admin] PATH_INFO: `/v3/projects/0c659656ba6b428586ccb1b86c017c95/users/f44128b890f34d4fbebfa828f4a37964/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:15.857594 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d19bb2-8fc8-4086-af6c-a22cee49c451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c659656ba6b428586ccb1b86c017c95, user_id=f44128b890f34d4fbebfa828f4a37964, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:15.860922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d19bb2-8fc8-4086-af6c-a22cee49c451 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:15.881579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1181/2351] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:15 2022] PUT /identity/v3/projects/0c659656ba6b428586ccb1b86c017c95/users/f44128b890f34d4fbebfa828f4a37964/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:00:15.894115 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f53e4-e74d-4b28-bbf4-7a611df829c1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:15.894405 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f53e4-e74d-4b28-bbf4-7a611df829c1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:15.894830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f53e4-e74d-4b28-bbf4-7a611df829c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:15.923783 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b99f53e4-e74d-4b28-bbf4-7a611df829c1 None None] MFA Rules not processed for user `f44128b890f34d4fbebfa828f4a37964`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:15.925014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b99f53e4-e74d-4b28-bbf4-7a611df829c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:15.992197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1171/2352] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:15.997022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d19bb2-8fc8-4086-af6c-a22cee49c451 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:16.061362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d19bb2-8fc8-4086-af6c-a22cee49c451 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:16.066080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a49089-692d-44fb-bd0d-4c6b1c9b321b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:16.066743 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a49089-692d-44fb-bd0d-4c6b1c9b321b admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:16.067014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a49089-692d-44fb-bd0d-4c6b1c9b321b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:16.067280 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a49089-692d-44fb-bd0d-4c6b1c9b321b admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:16.067960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a49089-692d-44fb-bd0d-4c6b1c9b321b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:16.069842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a49089-692d-44fb-bd0d-4c6b1c9b321b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:16.098555 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1182/2353] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:15 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 14:00:16.102865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f53e4-e74d-4b28-bbf4-7a611df829c1 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:16.116450 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f53e4-e74d-4b28-bbf4-7a611df829c1 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:16.141843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a49089-692d-44fb-bd0d-4c6b1c9b321b admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:16.143615 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-78a49089-692d-44fb-bd0d-4c6b1c9b321b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:16.157464 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:16.193840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696030e8-8a57-443f-aba9-a659d0387f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:16.194461 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696030e8-8a57-443f-aba9-a659d0387f75 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:16.194762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696030e8-8a57-443f-aba9-a659d0387f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:16.195003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696030e8-8a57-443f-aba9-a659d0387f75 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:16.195755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696030e8-8a57-443f-aba9-a659d0387f75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:16.197789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696030e8-8a57-443f-aba9-a659d0387f75 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:16.213705 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a49089-692d-44fb-bd0d-4c6b1c9b321b admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:16.218753 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06119a8b-cc77-42a3-b845-d55546bb668a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:16.219829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06119a8b-cc77-42a3-b845-d55546bb668a service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:16.219829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06119a8b-cc77-42a3-b845-d55546bb668a service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:16.220038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06119a8b-cc77-42a3-b845-d55546bb668a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:16.222312 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-06119a8b-cc77-42a3-b845-d55546bb668a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:16.227546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1172/2354] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:16 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:16.233463 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696030e8-8a57-443f-aba9-a659d0387f75 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:16.236625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06119a8b-cc77-42a3-b845-d55546bb668a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:16.238090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06119a8b-cc77-42a3-b845-d55546bb668a service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:16.245699 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696030e8-8a57-443f-aba9-a659d0387f75 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:16.247434 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1183/2355] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 14:00:16.250749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924d974-f6ca-46cf-ac4e-9378cf4a5377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:16.251522 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924d974-f6ca-46cf-ac4e-9378cf4a5377 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:16.251904 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924d974-f6ca-46cf-ac4e-9378cf4a5377 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:16.252295 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924d974-f6ca-46cf-ac4e-9378cf4a5377 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:16.253016 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d924d974-f6ca-46cf-ac4e-9378cf4a5377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:16.255021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d924d974-f6ca-46cf-ac4e-9378cf4a5377 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:16.266449 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1173/2356] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 14:00:16.275405 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06119a8b-cc77-42a3-b845-d55546bb668a service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:16.283626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06119a8b-cc77-42a3-b845-d55546bb668a service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:16.293264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f378046-8716-4346-a0b6-a732c6cc5ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:16.293264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f378046-8716-4346-a0b6-a732c6cc5ffc admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:16.293264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f378046-8716-4346-a0b6-a732c6cc5ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:16.293264 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f378046-8716-4346-a0b6-a732c6cc5ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:16.293717 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f378046-8716-4346-a0b6-a732c6cc5ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:16.294491 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f378046-8716-4346-a0b6-a732c6cc5ffc admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:16.305300 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1184/2357] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:16.309866 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924d974-f6ca-46cf-ac4e-9378cf4a5377 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:16.321241 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924d974-f6ca-46cf-ac4e-9378cf4a5377 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:16.326657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b67430-2b0d-4065-8d12-681dd53748dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:16.327324 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b67430-2b0d-4065-8d12-681dd53748dd admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:16.327595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b67430-2b0d-4065-8d12-681dd53748dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:16.327971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b67430-2b0d-4065-8d12-681dd53748dd admin admin] PATH_INFO: `/v3/projects/b42adedff67b4a85896fee855d92d0b2/users/487fecd6467c4f0895411e60cf0a1612/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:16.344796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b67430-2b0d-4065-8d12-681dd53748dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42adedff67b4a85896fee855d92d0b2, user_id=487fecd6467c4f0895411e60cf0a1612, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:16.346954 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b67430-2b0d-4065-8d12-681dd53748dd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:16.364654 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1174/2358] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:16 2022] PUT /identity/v3/projects/b42adedff67b4a85896fee855d92d0b2/users/487fecd6467c4f0895411e60cf0a1612/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:16.375543 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85e19a-c2cb-472d-a151-f70fd14cb09e None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:16.375968 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85e19a-c2cb-472d-a151-f70fd14cb09e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:16.376094 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85e19a-c2cb-472d-a151-f70fd14cb09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:16.406850 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-1d85e19a-c2cb-472d-a151-f70fd14cb09e None None] MFA Rules not processed for user `487fecd6467c4f0895411e60cf0a1612`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:16.408461 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1d85e19a-c2cb-472d-a151-f70fd14cb09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:16.484563 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1185/2359] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:16.489037 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b67430-2b0d-4065-8d12-681dd53748dd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:16.540747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:16.564098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b67430-2b0d-4065-8d12-681dd53748dd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:16.569884 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98b322-5418-436c-8200-75a43120f064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:16.570598 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98b322-5418-436c-8200-75a43120f064 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:16.570885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98b322-5418-436c-8200-75a43120f064 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:16.571151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98b322-5418-436c-8200-75a43120f064 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:16.572053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98b322-5418-436c-8200-75a43120f064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:16.573979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98b322-5418-436c-8200-75a43120f064 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:16.612084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1175/2360] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:16 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:16.615555 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85e19a-c2cb-472d-a151-f70fd14cb09e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:16.628159 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85e19a-c2cb-472d-a151-f70fd14cb09e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:16.692358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd1e40-043a-4bc2-bb4f-e0a6665033e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:16.693090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd1e40-043a-4bc2-bb4f-e0a6665033e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:16.693443 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd1e40-043a-4bc2-bb4f-e0a6665033e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:16.693815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd1e40-043a-4bc2-bb4f-e0a6665033e5 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:16.694546 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd1e40-043a-4bc2-bb4f-e0a6665033e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:16.696947 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd1e40-043a-4bc2-bb4f-e0a6665033e5 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:16.724340 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1186/2361] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:16 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:16.725548 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98b322-5418-436c-8200-75a43120f064 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:16.737664 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98b322-5418-436c-8200-75a43120f064 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:16.743527 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdd750-a0c2-46a2-b275-7304bc61eacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:16.752574 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdd750-a0c2-46a2-b275-7304bc61eacc admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:16.752878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdd750-a0c2-46a2-b275-7304bc61eacc admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:16.753142 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdd750-a0c2-46a2-b275-7304bc61eacc admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:16.753947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cdd750-a0c2-46a2-b275-7304bc61eacc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:16.755762 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cdd750-a0c2-46a2-b275-7304bc61eacc admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:16.769769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1176/2362] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:16.774166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd1e40-043a-4bc2-bb4f-e0a6665033e5 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:16.786155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd1e40-043a-4bc2-bb4f-e0a6665033e5 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:16.791279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234721f-e1ca-4692-b4a3-9f9159c8b200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:16.792154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234721f-e1ca-4692-b4a3-9f9159c8b200 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:16.792534 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234721f-e1ca-4692-b4a3-9f9159c8b200 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:16.792917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234721f-e1ca-4692-b4a3-9f9159c8b200 admin admin] PATH_INFO: `/v3/projects/eab0d4cc8bab479498c01211f66ea495/users/a4b4205cb10946cea1c27b1c37ae8a57/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:16.811422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234721f-e1ca-4692-b4a3-9f9159c8b200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab0d4cc8bab479498c01211f66ea495, user_id=a4b4205cb10946cea1c27b1c37ae8a57, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:16.811641 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdd750-a0c2-46a2-b275-7304bc61eacc admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:16.813653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e2cdd750-a0c2-46a2-b275-7304bc61eacc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:16.814437 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234721f-e1ca-4692-b4a3-9f9159c8b200 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:16.827050 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:16.841843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1187/2363] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:16 2022] PUT /identity/v3/projects/eab0d4cc8bab479498c01211f66ea495/users/a4b4205cb10946cea1c27b1c37ae8a57/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:16.845684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234721f-e1ca-4692-b4a3-9f9159c8b200 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:16.916166 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdd750-a0c2-46a2-b275-7304bc61eacc admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:16.935263 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234721f-e1ca-4692-b4a3-9f9159c8b200 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:16.940582 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c33379-5fa5-4806-8d44-72ef2bc8080e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:16.941222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c33379-5fa5-4806-8d44-72ef2bc8080e admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:16.941508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c33379-5fa5-4806-8d44-72ef2bc8080e admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:16.941779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c33379-5fa5-4806-8d44-72ef2bc8080e admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:16.942559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c33379-5fa5-4806-8d44-72ef2bc8080e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:16.944421 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c33379-5fa5-4806-8d44-72ef2bc8080e admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:16.966465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1188/2364] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:16.967569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c33379-5fa5-4806-8d44-72ef2bc8080e admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:16.978737 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c33379-5fa5-4806-8d44-72ef2bc8080e admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:16.985625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b40e0c-85f0-4bd9-b74e-c8312669be97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:16.986462 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b40e0c-85f0-4bd9-b74e-c8312669be97 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:16.986751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b40e0c-85f0-4bd9-b74e-c8312669be97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:16.987027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b40e0c-85f0-4bd9-b74e-c8312669be97 admin admin] PATH_INFO: `/v3/projects/eab0d4cc8bab479498c01211f66ea495/users/a4b4205cb10946cea1c27b1c37ae8a57/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:16.989052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b40e0c-85f0-4bd9-b74e-c8312669be97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab0d4cc8bab479498c01211f66ea495, user_id=a4b4205cb10946cea1c27b1c37ae8a57, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:16.989567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5b53c-b406-430a-83ec-90bcd437c574 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6720518c03413aa8f85e4d6142fe87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:16.990207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5b53c-b406-430a-83ec-90bcd437c574 service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:16.990837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5b53c-b406-430a-83ec-90bcd437c574 service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:16.991136 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5b53c-b406-430a-83ec-90bcd437c574 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:16.992380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b40e0c-85f0-4bd9-b74e-c8312669be97 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:16.993513 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-02b5b53c-b406-430a-83ec-90bcd437c574 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:17.006498 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5b53c-b406-430a-83ec-90bcd437c574 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:17.008233 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5b53c-b406-430a-83ec-90bcd437c574 service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:17.008760 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1189/2365] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:16 2022] PUT /identity/v3/projects/eab0d4cc8bab479498c01211f66ea495/users/a4b4205cb10946cea1c27b1c37ae8a57/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:17.020069 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e871e9d-7d01-49ce-be6e-0a6d3e617231 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:17.020574 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e871e9d-7d01-49ce-be6e-0a6d3e617231 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:17.020875 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e871e9d-7d01-49ce-be6e-0a6d3e617231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:17.029902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1177/2366] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:17.063957 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-3e871e9d-7d01-49ce-be6e-0a6d3e617231 None None] MFA Rules not processed for user `a4b4205cb10946cea1c27b1c37ae8a57`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:17.065465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3e871e9d-7d01-49ce-be6e-0a6d3e617231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:17.148689 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1190/2367] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:17 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:17.162336 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5b53c-b406-430a-83ec-90bcd437c574 service placement] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:17.164247 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-02b5b53c-b406-430a-83ec-90bcd437c574 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:17.223812 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:17.228859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5b53c-b406-430a-83ec-90bcd437c574 service placement] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:17.285228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bcb70-b414-4b19-89a9-9310c254bd9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:17.285839 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bcb70-b414-4b19-89a9-9310c254bd9a service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:17.286111 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bcb70-b414-4b19-89a9-9310c254bd9a service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:17.286374 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bcb70-b414-4b19-89a9-9310c254bd9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:17.296842 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58bcb70-b414-4b19-89a9-9310c254bd9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:17.298906 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58bcb70-b414-4b19-89a9-9310c254bd9a service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:17.319628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1178/2368] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 14:00:17.854939 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e871e9d-7d01-49ce-be6e-0a6d3e617231 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:17.856428 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3e871e9d-7d01-49ce-be6e-0a6d3e617231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:17.869141 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e871e9d-7d01-49ce-be6e-0a6d3e617231 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:17.873444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19e25d-8b26-4ad8-8491-0c13a4686e19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:17.874052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19e25d-8b26-4ad8-8491-0c13a4686e19 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:17.874324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19e25d-8b26-4ad8-8491-0c13a4686e19 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:17.874590 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19e25d-8b26-4ad8-8491-0c13a4686e19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:17.940864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19e25d-8b26-4ad8-8491-0c13a4686e19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:17.942313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19e25d-8b26-4ad8-8491-0c13a4686e19 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:17.951825 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1191/2369] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:18.448333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bcb70-b414-4b19-89a9-9310c254bd9a service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:18.449785 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a58bcb70-b414-4b19-89a9-9310c254bd9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:18.518491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bcb70-b414-4b19-89a9-9310c254bd9a service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:18.523061 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b500be-3c79-4269-ba76-c710803348a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:18.523701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b500be-3c79-4269-ba76-c710803348a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:18.524019 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b500be-3c79-4269-ba76-c710803348a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:18.524291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b500be-3c79-4269-ba76-c710803348a4 admin admin] PATH_INFO: `/v3/users/31c3b0fc07d34cfe94bda83f5993daa7` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:18.525439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b500be-3c79-4269-ba76-c710803348a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c3b0fc07d34cfe94bda83f5993daa7)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:18.527445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b500be-3c79-4269-ba76-c710803348a4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:18.575961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-16b500be-3c79-4269-ba76-c710803348a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c3b0fc07d34cfe94bda83f5993daa7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:18.576624 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-16b500be-3c79-4269-ba76-c710803348a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c3b0fc07d34cfe94bda83f5993daa7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:18.577206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-16b500be-3c79-4269-ba76-c710803348a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c3b0fc07d34cfe94bda83f5993daa7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:18.590597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-16b500be-3c79-4269-ba76-c710803348a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c3b0fc07d34cfe94bda83f5993daa7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:18.600457 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-16b500be-3c79-4269-ba76-c710803348a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c3b0fc07d34cfe94bda83f5993daa7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:18.613957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1179/2370] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:18 2022] DELETE /identity/v3/users/31c3b0fc07d34cfe94bda83f5993daa7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:18.618735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aab37b-4add-42d7-bf41-6fb6286a8b42 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:18.619008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aab37b-4add-42d7-bf41-6fb6286a8b42 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:18.619270 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aab37b-4add-42d7-bf41-6fb6286a8b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:18.627959 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:18.641324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-46aab37b-4add-42d7-bf41-6fb6286a8b42 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:18.642629 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-46aab37b-4add-42d7-bf41-6fb6286a8b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:18.711525 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1192/2371] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:18.721417 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b500be-3c79-4269-ba76-c710803348a4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:18.723020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-16b500be-3c79-4269-ba76-c710803348a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:18.755157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:18.785134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b500be-3c79-4269-ba76-c710803348a4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:18.789021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed179a-c72b-4ee8-bbc7-ef2ed383c531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:18.789645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed179a-c72b-4ee8-bbc7-ef2ed383c531 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:18.789922 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed179a-c72b-4ee8-bbc7-ef2ed383c531 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:18.790190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed179a-c72b-4ee8-bbc7-ef2ed383c531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:18.800360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed179a-c72b-4ee8-bbc7-ef2ed383c531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:18.801836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed179a-c72b-4ee8-bbc7-ef2ed383c531 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:18.814159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1180/2372] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 14:00:19.130097 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aab37b-4add-42d7-bf41-6fb6286a8b42 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:19.131626 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-46aab37b-4add-42d7-bf41-6fb6286a8b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:19.145042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aab37b-4add-42d7-bf41-6fb6286a8b42 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:19.152000 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:19.152772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:19.153491 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:19.153918 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] PATH_INFO: `/v3/projects/0330fde8c21341ad95008a4386ac5080` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:19.155090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0330fde8c21341ad95008a4386ac5080)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:19.156979 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:19.223531 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0330fde8c21341ad95008a4386ac5080', 'request_id': 'req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:19.232986 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0330fde8c21341ad95008a4386ac5080', 'request_id': 'req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:19.241190 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0330fde8c21341ad95008a4386ac5080', 'request_id': 'req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:19.253178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] The token cache is being invalidate because project 0330fde8c21341ad95008a4386ac5080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:19.253559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:19.254413 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:19.256490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1193/2373] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:19 2022] DELETE /identity/v3/projects/0330fde8c21341ad95008a4386ac5080 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:19.261107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed179a-c72b-4ee8-bbc7-ef2ed383c531 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:19.272165 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed179a-c72b-4ee8-bbc7-ef2ed383c531 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:19.341167 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a15c37-2407-4e96-98fe-379768f90eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:19.342193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a15c37-2407-4e96-98fe-379768f90eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:19.342721 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a15c37-2407-4e96-98fe-379768f90eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:19.343429 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a15c37-2407-4e96-98fe-379768f90eb5 admin admin] PATH_INFO: `/v3/users/8cd0823776bd4168823236b119a18b34` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:19.345313 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a15c37-2407-4e96-98fe-379768f90eb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cd0823776bd4168823236b119a18b34)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:19.348656 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a15c37-2407-4e96-98fe-379768f90eb5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:19.445806 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a6a15c37-2407-4e96-98fe-379768f90eb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cd0823776bd4168823236b119a18b34'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:19.447714 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a6a15c37-2407-4e96-98fe-379768f90eb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cd0823776bd4168823236b119a18b34'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:19.448605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a6a15c37-2407-4e96-98fe-379768f90eb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cd0823776bd4168823236b119a18b34'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:19.477743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a6a15c37-2407-4e96-98fe-379768f90eb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cd0823776bd4168823236b119a18b34'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:19.488508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a6a15c37-2407-4e96-98fe-379768f90eb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cd0823776bd4168823236b119a18b34'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:19.514895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1181/2374] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:19 2022] DELETE /identity/v3/users/8cd0823776bd4168823236b119a18b34 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:19.971166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:19.972705 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:19.986124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db1f75-ae20-4df6-b6fe-2cc0f9595de8 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:20.052847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb488dd-f2e2-40c3-b580-e11d190d19f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:20.053574 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb488dd-f2e2-40c3-b580-e11d190d19f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:20.053842 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb488dd-f2e2-40c3-b580-e11d190d19f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:20.054114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb488dd-f2e2-40c3-b580-e11d190d19f1 admin admin] PATH_INFO: `/v3/projects/ac269cd714724d6f9d82cb85f748ba79` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:20.055373 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb488dd-f2e2-40c3-b580-e11d190d19f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac269cd714724d6f9d82cb85f748ba79)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:20.057330 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb488dd-f2e2-40c3-b580-e11d190d19f1 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:20.094601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:20.110480 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9bb488dd-f2e2-40c3-b580-e11d190d19f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac269cd714724d6f9d82cb85f748ba79', 'request_id': 'req-9bb488dd-f2e2-40c3-b580-e11d190d19f1'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:20.119409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9bb488dd-f2e2-40c3-b580-e11d190d19f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac269cd714724d6f9d82cb85f748ba79', 'request_id': 'req-9bb488dd-f2e2-40c3-b580-e11d190d19f1'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:20.126454 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9bb488dd-f2e2-40c3-b580-e11d190d19f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac269cd714724d6f9d82cb85f748ba79', 'request_id': 'req-9bb488dd-f2e2-40c3-b580-e11d190d19f1'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:20.136941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9bb488dd-f2e2-40c3-b580-e11d190d19f1 admin admin] The token cache is being invalidate because project ac269cd714724d6f9d82cb85f748ba79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:20.137605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9bb488dd-f2e2-40c3-b580-e11d190d19f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:20.137889 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9bb488dd-f2e2-40c3-b580-e11d190d19f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:20.139387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1194/2375] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:19 2022] DELETE /identity/v3/projects/ac269cd714724d6f9d82cb85f748ba79 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:20.144858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464a864-b342-4c7c-870e-ad902033cb51 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:20.145086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464a864-b342-4c7c-870e-ad902033cb51 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:20.145358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464a864-b342-4c7c-870e-ad902033cb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:20.166192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8464a864-b342-4c7c-870e-ad902033cb51 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:20.167706 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8464a864-b342-4c7c-870e-ad902033cb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:20.216555 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:20.236247 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1182/2376] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 14:00:20.244222 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5feb8-049b-473b-98b6-f34e2c039b1b None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:20.244528 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5feb8-049b-473b-98b6-f34e2c039b1b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:20.244809 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5feb8-049b-473b-98b6-f34e2c039b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:20.264729 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-34c5feb8-049b-473b-98b6-f34e2c039b1b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:20.265968 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-34c5feb8-049b-473b-98b6-f34e2c039b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:20.274170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1195/2377] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:20.278610 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464a864-b342-4c7c-870e-ad902033cb51 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:20.289370 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464a864-b342-4c7c-870e-ad902033cb51 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:20.294958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf33ba-1c2a-4c7c-b705-7d1159f76fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:20.295678 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf33ba-1c2a-4c7c-b705-7d1159f76fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:20.295980 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf33ba-1c2a-4c7c-b705-7d1159f76fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:20.296245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf33ba-1c2a-4c7c-b705-7d1159f76fb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:20.296885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf33ba-1c2a-4c7c-b705-7d1159f76fb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:00:20.297176 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf33ba-1c2a-4c7c-b705-7d1159f76fb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:20.298811 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf33ba-1c2a-4c7c-b705-7d1159f76fb1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:20.311336 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1183/2378] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 14:00:20.316190 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5feb8-049b-473b-98b6-f34e2c039b1b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:20.327075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5feb8-049b-473b-98b6-f34e2c039b1b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:20.332080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f936109-1014-486c-95bb-5a41540cb0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:20.332702 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f936109-1014-486c-95bb-5a41540cb0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:20.332981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f936109-1014-486c-95bb-5a41540cb0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:20.333254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f936109-1014-486c-95bb-5a41540cb0c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:20.333898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f936109-1014-486c-95bb-5a41540cb0c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:20.335811 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f936109-1014-486c-95bb-5a41540cb0c6 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:20.367054 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1196/2379] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:20 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:20.372261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf33ba-1c2a-4c7c-b705-7d1159f76fb1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:20.384460 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf33ba-1c2a-4c7c-b705-7d1159f76fb1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:20.453052 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf278ef-a170-452f-9416-8893247449b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:20.453738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf278ef-a170-452f-9416-8893247449b6 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:20.454210 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf278ef-a170-452f-9416-8893247449b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:20.454210 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf278ef-a170-452f-9416-8893247449b6 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:20.454752 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf278ef-a170-452f-9416-8893247449b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:20.457178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf278ef-a170-452f-9416-8893247449b6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:20.487605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1184/2380] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:20 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:20.492061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f936109-1014-486c-95bb-5a41540cb0c6 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:20.503167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f936109-1014-486c-95bb-5a41540cb0c6 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:20.508274 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57410b10-49c6-4528-947d-8333c23aeef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:20.508886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57410b10-49c6-4528-947d-8333c23aeef0 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:20.509158 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57410b10-49c6-4528-947d-8333c23aeef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:20.509427 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57410b10-49c6-4528-947d-8333c23aeef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:20.510067 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57410b10-49c6-4528-947d-8333c23aeef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:20.511810 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57410b10-49c6-4528-947d-8333c23aeef0 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:20.522294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1197/2381] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 14:00:20.527937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf278ef-a170-452f-9416-8893247449b6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:20.538366 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf278ef-a170-452f-9416-8893247449b6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:20.543182 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89758596-fe42-4256-b030-f25d2497f3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:20.543796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89758596-fe42-4256-b030-f25d2497f3bc admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:20.544099 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89758596-fe42-4256-b030-f25d2497f3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:20.544364 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89758596-fe42-4256-b030-f25d2497f3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:20.544996 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89758596-fe42-4256-b030-f25d2497f3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:20.546599 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89758596-fe42-4256-b030-f25d2497f3bc admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:20.557042 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1185/2382] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:20.561345 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57410b10-49c6-4528-947d-8333c23aeef0 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:20.572372 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57410b10-49c6-4528-947d-8333c23aeef0 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:20.577683 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862c09a-45e5-4dd7-828e-d759783f73e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:20.578332 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862c09a-45e5-4dd7-828e-d759783f73e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:20.578609 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862c09a-45e5-4dd7-828e-d759783f73e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:20.578870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862c09a-45e5-4dd7-828e-d759783f73e6 admin admin] PATH_INFO: `/v3/projects/62c7896dfbe44ea48236b9a6a63a9621/users/be34297b9b4f48409dfb50e3af883b92/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:20.596473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862c09a-45e5-4dd7-828e-d759783f73e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62c7896dfbe44ea48236b9a6a63a9621, user_id=be34297b9b4f48409dfb50e3af883b92, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:20.600712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862c09a-45e5-4dd7-828e-d759783f73e6 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:20.633864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1198/2383] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:20 2022] PUT /identity/v3/projects/62c7896dfbe44ea48236b9a6a63a9621/users/be34297b9b4f48409dfb50e3af883b92/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:20.645553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0a6c1-31c0-4b42-8324-465f2402241d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:20.645824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0a6c1-31c0-4b42-8324-465f2402241d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:20.646095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0a6c1-31c0-4b42-8324-465f2402241d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:20.677674 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-27c0a6c1-31c0-4b42-8324-465f2402241d None None] MFA Rules not processed for user `be34297b9b4f48409dfb50e3af883b92`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:20.678971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-27c0a6c1-31c0-4b42-8324-465f2402241d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:20.766051 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1186/2384] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:20.770728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862c09a-45e5-4dd7-828e-d759783f73e6 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:20.838002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862c09a-45e5-4dd7-828e-d759783f73e6 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:20.843010 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240cdcf-f948-425e-af68-1687f4d18970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:20.843778 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240cdcf-f948-425e-af68-1687f4d18970 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:20.844167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240cdcf-f948-425e-af68-1687f4d18970 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:20.844529 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240cdcf-f948-425e-af68-1687f4d18970 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:20.845265 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240cdcf-f948-425e-af68-1687f4d18970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:20.847205 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240cdcf-f948-425e-af68-1687f4d18970 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:20.889283 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1199/2385] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:20 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:20.889984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0a6c1-31c0-4b42-8324-465f2402241d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:20.901325 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0a6c1-31c0-4b42-8324-465f2402241d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:20.952666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:20.969877 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434aac98-a276-40e5-83c4-eda6b7ecf1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:20.970470 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434aac98-a276-40e5-83c4-eda6b7ecf1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:20.970740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434aac98-a276-40e5-83c4-eda6b7ecf1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:20.971051 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434aac98-a276-40e5-83c4-eda6b7ecf1e8 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:20.971626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434aac98-a276-40e5-83c4-eda6b7ecf1e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:20.973561 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434aac98-a276-40e5-83c4-eda6b7ecf1e8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:20.996503 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1187/2386] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:20 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 14:00:21.000742 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240cdcf-f948-425e-af68-1687f4d18970 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:21.011479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240cdcf-f948-425e-af68-1687f4d18970 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:21.016559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f797368-2da9-4671-b21a-0daa4d41368d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:21.017166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f797368-2da9-4671-b21a-0daa4d41368d admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:21.017444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f797368-2da9-4671-b21a-0daa4d41368d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:21.017774 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f797368-2da9-4671-b21a-0daa4d41368d admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:21.018409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f797368-2da9-4671-b21a-0daa4d41368d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:21.020528 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f797368-2da9-4671-b21a-0daa4d41368d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:21.031774 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1200/2387] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:21.037038 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434aac98-a276-40e5-83c4-eda6b7ecf1e8 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:21.047798 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434aac98-a276-40e5-83c4-eda6b7ecf1e8 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:21.053694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d103727-780d-487e-8354-48b0af99fee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:21.054306 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d103727-780d-487e-8354-48b0af99fee1 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:21.054584 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d103727-780d-487e-8354-48b0af99fee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:21.054852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d103727-780d-487e-8354-48b0af99fee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:21.055487 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d103727-780d-487e-8354-48b0af99fee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:21.057154 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d103727-780d-487e-8354-48b0af99fee1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:21.068011 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1188/2388] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 14:00:21.072267 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f797368-2da9-4671-b21a-0daa4d41368d admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:21.083239 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f797368-2da9-4671-b21a-0daa4d41368d admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:21.089188 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2cbd6-b33c-466e-acca-73f22f9114b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:21.089998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2cbd6-b33c-466e-acca-73f22f9114b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:21.090283 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2cbd6-b33c-466e-acca-73f22f9114b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:21.090556 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2cbd6-b33c-466e-acca-73f22f9114b4 admin admin] PATH_INFO: `/v3/projects/61e0be896e5a49459cf7c2ffab05e19a/users/451ddc5891d14be99787aedb541ccfe3/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:21.107321 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2cbd6-b33c-466e-acca-73f22f9114b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e0be896e5a49459cf7c2ffab05e19a, user_id=451ddc5891d14be99787aedb541ccfe3, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:21.110130 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2cbd6-b33c-466e-acca-73f22f9114b4 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:21.126986 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1201/2389] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:21 2022] PUT /identity/v3/projects/61e0be896e5a49459cf7c2ffab05e19a/users/451ddc5891d14be99787aedb541ccfe3/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:21.138943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715ce30-271f-4afe-8a4a-33b5ea62d900 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:21.139333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715ce30-271f-4afe-8a4a-33b5ea62d900 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:21.139673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715ce30-271f-4afe-8a4a-33b5ea62d900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:21.169276 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a715ce30-271f-4afe-8a4a-33b5ea62d900 None None] MFA Rules not processed for user `451ddc5891d14be99787aedb541ccfe3`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:21.170487 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a715ce30-271f-4afe-8a4a-33b5ea62d900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:21.248583 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1189/2390] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 14:00:21.253939 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2cbd6-b33c-466e-acca-73f22f9114b4 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:21.260351 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:21.330605 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2cbd6-b33c-466e-acca-73f22f9114b4 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:21.335576 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba52c32-7afb-4c2d-8645-463846709663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:21.336215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba52c32-7afb-4c2d-8645-463846709663 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:21.336495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba52c32-7afb-4c2d-8645-463846709663 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:21.336764 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba52c32-7afb-4c2d-8645-463846709663 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:21.337395 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba52c32-7afb-4c2d-8645-463846709663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:21.339634 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba52c32-7afb-4c2d-8645-463846709663 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:21.375459 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1202/2391] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:21 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 14:00:21.377055 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715ce30-271f-4afe-8a4a-33b5ea62d900 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:21.389084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715ce30-271f-4afe-8a4a-33b5ea62d900 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:21.451389 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ef806-1543-45a3-b1ca-01c27f3a84b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:21.452091 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ef806-1543-45a3-b1ca-01c27f3a84b1 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:21.452312 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ef806-1543-45a3-b1ca-01c27f3a84b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:21.452577 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ef806-1543-45a3-b1ca-01c27f3a84b1 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:21.453203 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ef806-1543-45a3-b1ca-01c27f3a84b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:21.455145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ef806-1543-45a3-b1ca-01c27f3a84b1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:21.480181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1190/2392] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:21 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 14:00:21.484215 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba52c32-7afb-4c2d-8645-463846709663 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:21.495402 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba52c32-7afb-4c2d-8645-463846709663 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:21.501226 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1eb3c-3973-47fe-8573-52971572f5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:21.501734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1eb3c-3973-47fe-8573-52971572f5be admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:21.502004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1eb3c-3973-47fe-8573-52971572f5be admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:21.502265 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1eb3c-3973-47fe-8573-52971572f5be admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:21.502908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1eb3c-3973-47fe-8573-52971572f5be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:21.504553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1eb3c-3973-47fe-8573-52971572f5be admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:21.515933 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1203/2393] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:21.520251 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ef806-1543-45a3-b1ca-01c27f3a84b1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:21.531136 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ef806-1543-45a3-b1ca-01c27f3a84b1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:21.535943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced5f71-d26d-4cb7-b391-7ddc82f7d68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:21.536593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced5f71-d26d-4cb7-b391-7ddc82f7d68b admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:21.536869 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced5f71-d26d-4cb7-b391-7ddc82f7d68b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:21.537119 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced5f71-d26d-4cb7-b391-7ddc82f7d68b admin admin] PATH_INFO: `/v3/projects/e07a1c835d244b2286582627e0787be6/users/53982ea779884c5692ad598f29269807/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:21.552495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced5f71-d26d-4cb7-b391-7ddc82f7d68b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07a1c835d244b2286582627e0787be6, user_id=53982ea779884c5692ad598f29269807, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:21.555159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced5f71-d26d-4cb7-b391-7ddc82f7d68b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:21.571851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1191/2394] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:21 2022] PUT /identity/v3/projects/e07a1c835d244b2286582627e0787be6/users/53982ea779884c5692ad598f29269807/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:21.576284 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1eb3c-3973-47fe-8573-52971572f5be admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:21.586778 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1eb3c-3973-47fe-8573-52971572f5be admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:21.644656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61aca77-f226-4fc1-a9a3-6b17206669a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:21.645483 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aca77-f226-4fc1-a9a3-6b17206669a0 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:21.645820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aca77-f226-4fc1-a9a3-6b17206669a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:21.646203 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aca77-f226-4fc1-a9a3-6b17206669a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:21.646938 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61aca77-f226-4fc1-a9a3-6b17206669a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:21.648669 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61aca77-f226-4fc1-a9a3-6b17206669a0 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:21.660286 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1204/2395] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 14:00:21.664610 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced5f71-d26d-4cb7-b391-7ddc82f7d68b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:21.676135 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced5f71-d26d-4cb7-b391-7ddc82f7d68b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:21.682359 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38cdaf-8d9b-4be4-9cee-9cff3f5b49fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:21.683352 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38cdaf-8d9b-4be4-9cee-9cff3f5b49fb admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:21.683663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38cdaf-8d9b-4be4-9cee-9cff3f5b49fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:21.684081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38cdaf-8d9b-4be4-9cee-9cff3f5b49fb admin admin] PATH_INFO: `/v3/projects/e07a1c835d244b2286582627e0787be6/users/53982ea779884c5692ad598f29269807/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:21.686718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38cdaf-8d9b-4be4-9cee-9cff3f5b49fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07a1c835d244b2286582627e0787be6, user_id=53982ea779884c5692ad598f29269807, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:21.689414 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38cdaf-8d9b-4be4-9cee-9cff3f5b49fb admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:21.700322 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1192/2396] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:21 2022] PUT /identity/v3/projects/e07a1c835d244b2286582627e0787be6/users/53982ea779884c5692ad598f29269807/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:21.711546 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faaba3-2a15-4493-89ad-df5e64585b6b None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:21.711837 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faaba3-2a15-4493-89ad-df5e64585b6b None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:21.712172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faaba3-2a15-4493-89ad-df5e64585b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:21.742268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-13faaba3-2a15-4493-89ad-df5e64585b6b None None] MFA Rules not processed for user `53982ea779884c5692ad598f29269807`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:21.743519 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-13faaba3-2a15-4493-89ad-df5e64585b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:21.825342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:21.836251 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1205/2397] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:21.843688 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38cdaf-8d9b-4be4-9cee-9cff3f5b49fb admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:21.845494 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3b38cdaf-8d9b-4be4-9cee-9cff3f5b49fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:21.852810 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:21.916221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38cdaf-8d9b-4be4-9cee-9cff3f5b49fb admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:21.920052 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16986878-2995-4b35-b7ec-538e9504b1d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:21.920736 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16986878-2995-4b35-b7ec-538e9504b1d0 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:21.921081 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16986878-2995-4b35-b7ec-538e9504b1d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:21.921418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16986878-2995-4b35-b7ec-538e9504b1d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:21.930865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16986878-2995-4b35-b7ec-538e9504b1d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:21.932395 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16986878-2995-4b35-b7ec-538e9504b1d0 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:21.942817 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1193/2398] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:21.993555 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faaba3-2a15-4493-89ad-df5e64585b6b None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:22.006268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faaba3-2a15-4493-89ad-df5e64585b6b None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:22.010149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c4a08-9c19-4d62-8cbe-01254c3d1ae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:22.010892 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c4a08-9c19-4d62-8cbe-01254c3d1ae5 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:22.011185 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c4a08-9c19-4d62-8cbe-01254c3d1ae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:22.011440 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c4a08-9c19-4d62-8cbe-01254c3d1ae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:22.069755 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c4a08-9c19-4d62-8cbe-01254c3d1ae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:22.071197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c4a08-9c19-4d62-8cbe-01254c3d1ae5 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:22.081456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1206/2399] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:22.177051 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16986878-2995-4b35-b7ec-538e9504b1d0 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:22.178439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-16986878-2995-4b35-b7ec-538e9504b1d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:22.189440 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16986878-2995-4b35-b7ec-538e9504b1d0 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:22.193262 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905ca43-8b37-4a5a-a56e-25b7ed3a48ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:22.193874 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905ca43-8b37-4a5a-a56e-25b7ed3a48ed service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:22.194191 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905ca43-8b37-4a5a-a56e-25b7ed3a48ed service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:22.194423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905ca43-8b37-4a5a-a56e-25b7ed3a48ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:22.196145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9905ca43-8b37-4a5a-a56e-25b7ed3a48ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:22.262438 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905ca43-8b37-4a5a-a56e-25b7ed3a48ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:22.263812 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905ca43-8b37-4a5a-a56e-25b7ed3a48ed service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:22.275452 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1194/2400] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:24.551262 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c4a08-9c19-4d62-8cbe-01254c3d1ae5 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:24.552864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-d97c4a08-9c19-4d62-8cbe-01254c3d1ae5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:24.622399 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c4a08-9c19-4d62-8cbe-01254c3d1ae5 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:24.627367 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8699548-0cb1-4394-8d4a-f4909fee2771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:24.628033 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8699548-0cb1-4394-8d4a-f4909fee2771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:24.628309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8699548-0cb1-4394-8d4a-f4909fee2771 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:24.628572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8699548-0cb1-4394-8d4a-f4909fee2771 admin admin] PATH_INFO: `/v3/users/f44128b890f34d4fbebfa828f4a37964` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:24.629839 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8699548-0cb1-4394-8d4a-f4909fee2771 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f44128b890f34d4fbebfa828f4a37964)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:24.631912 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8699548-0cb1-4394-8d4a-f4909fee2771 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:24.688979 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f8699548-0cb1-4394-8d4a-f4909fee2771 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f44128b890f34d4fbebfa828f4a37964'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:24.696078 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f8699548-0cb1-4394-8d4a-f4909fee2771 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f44128b890f34d4fbebfa828f4a37964'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:24.696777 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f8699548-0cb1-4394-8d4a-f4909fee2771 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f44128b890f34d4fbebfa828f4a37964'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:24.697431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f8699548-0cb1-4394-8d4a-f4909fee2771 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f44128b890f34d4fbebfa828f4a37964'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:24.707926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f8699548-0cb1-4394-8d4a-f4909fee2771 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f44128b890f34d4fbebfa828f4a37964'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:24.726226 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1207/2401] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:24 2022] DELETE /identity/v3/users/f44128b890f34d4fbebfa828f4a37964 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:24.731216 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dba94-e87a-46ae-82d8-717be7869ccb None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:24.731720 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dba94-e87a-46ae-82d8-717be7869ccb None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:24.731835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dba94-e87a-46ae-82d8-717be7869ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:24.752218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7b3dba94-e87a-46ae-82d8-717be7869ccb None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:24.753465 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7b3dba94-e87a-46ae-82d8-717be7869ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:24.822029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1195/2402] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:24.834227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8699548-0cb1-4394-8d4a-f4909fee2771 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:24.835707 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f8699548-0cb1-4394-8d4a-f4909fee2771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:24.857148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:24.896830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8699548-0cb1-4394-8d4a-f4909fee2771 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:24.901128 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295405ee-82c3-4309-b444-1c23107b2d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:24.901737 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295405ee-82c3-4309-b444-1c23107b2d23 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:24.902085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295405ee-82c3-4309-b444-1c23107b2d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:24.902277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295405ee-82c3-4309-b444-1c23107b2d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:24.911398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295405ee-82c3-4309-b444-1c23107b2d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:24.912864 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295405ee-82c3-4309-b444-1c23107b2d23 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:24.922193 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1208/2403] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:25.048304 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dba94-e87a-46ae-82d8-717be7869ccb None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:25.049741 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7b3dba94-e87a-46ae-82d8-717be7869ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:25.061593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dba94-e87a-46ae-82d8-717be7869ccb None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:25.066409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62bbc2-04e8-4912-b9d7-8ffa9c8beac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:25.067057 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62bbc2-04e8-4912-b9d7-8ffa9c8beac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:25.067337 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62bbc2-04e8-4912-b9d7-8ffa9c8beac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:25.067605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62bbc2-04e8-4912-b9d7-8ffa9c8beac9 admin admin] PATH_INFO: `/v3/users/be34297b9b4f48409dfb50e3af883b92` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:25.068800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62bbc2-04e8-4912-b9d7-8ffa9c8beac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be34297b9b4f48409dfb50e3af883b92)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:25.072080 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62bbc2-04e8-4912-b9d7-8ffa9c8beac9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:25.117566 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:25.125221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4e62bbc2-04e8-4912-b9d7-8ffa9c8beac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be34297b9b4f48409dfb50e3af883b92'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.125937 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4e62bbc2-04e8-4912-b9d7-8ffa9c8beac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be34297b9b4f48409dfb50e3af883b92'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.126936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4e62bbc2-04e8-4912-b9d7-8ffa9c8beac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be34297b9b4f48409dfb50e3af883b92'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.140683 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4e62bbc2-04e8-4912-b9d7-8ffa9c8beac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be34297b9b4f48409dfb50e3af883b92'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.149928 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4e62bbc2-04e8-4912-b9d7-8ffa9c8beac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be34297b9b4f48409dfb50e3af883b92'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.164021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1196/2404] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:25 2022] DELETE /identity/v3/users/be34297b9b4f48409dfb50e3af883b92 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:25.176379 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3a4c7-6846-4fb2-bffe-11d3187399ae None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:25.176746 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3a4c7-6846-4fb2-bffe-11d3187399ae None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:25.177089 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3a4c7-6846-4fb2-bffe-11d3187399ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:25.197630 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-8bb3a4c7-6846-4fb2-bffe-11d3187399ae None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:25.198847 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8bb3a4c7-6846-4fb2-bffe-11d3187399ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:25.210439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62bbc2-04e8-4912-b9d7-8ffa9c8beac9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:25.212124 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4e62bbc2-04e8-4912-b9d7-8ffa9c8beac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:25.269043 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1209/2405] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:25.278984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3a4c7-6846-4fb2-bffe-11d3187399ae None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:25.280544 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8bb3a4c7-6846-4fb2-bffe-11d3187399ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:25.288437 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62bbc2-04e8-4912-b9d7-8ffa9c8beac9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:25.293563 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:25.294212 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:25.294501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:25.294761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] PATH_INFO: `/v3/projects/0c659656ba6b428586ccb1b86c017c95` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:25.295978 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c659656ba6b428586ccb1b86c017c95)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:25.297947 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:25.339186 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3a4c7-6846-4fb2-bffe-11d3187399ae None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:25.346830 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c659656ba6b428586ccb1b86c017c95', 'request_id': 'req-3a734d41-e9f8-43f3-8009-eb38229541df'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.353855 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c659656ba6b428586ccb1b86c017c95', 'request_id': 'req-3a734d41-e9f8-43f3-8009-eb38229541df'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.362768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c659656ba6b428586ccb1b86c017c95', 'request_id': 'req-3a734d41-e9f8-43f3-8009-eb38229541df'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.373846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] The token cache is being invalidate because project 0c659656ba6b428586ccb1b86c017c95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:25.374038 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.374704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.376302 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1197/2406] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:25 2022] DELETE /identity/v3/projects/0c659656ba6b428586ccb1b86c017c95 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:25.381986 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:25.383524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:25.393675 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16de240-19b0-4362-97d3-f9f35ee7e3a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:25.394306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16de240-19b0-4362-97d3-f9f35ee7e3a7 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:25.395359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16de240-19b0-4362-97d3-f9f35ee7e3a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:25.395359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16de240-19b0-4362-97d3-f9f35ee7e3a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:25.397074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b16de240-19b0-4362-97d3-f9f35ee7e3a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:25.433678 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:25.434560 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:25.452311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a734d41-e9f8-43f3-8009-eb38229541df admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:25.459689 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666e2d4-6123-4b23-87a0-0cde84ff3c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:25.460491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666e2d4-6123-4b23-87a0-0cde84ff3c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:25.460914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666e2d4-6123-4b23-87a0-0cde84ff3c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:25.461195 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666e2d4-6123-4b23-87a0-0cde84ff3c3c admin admin] PATH_INFO: `/v3/users/487fecd6467c4f0895411e60cf0a1612` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:25.462462 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666e2d4-6123-4b23-87a0-0cde84ff3c3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=487fecd6467c4f0895411e60cf0a1612)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:25.464495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666e2d4-6123-4b23-87a0-0cde84ff3c3c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:25.473785 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16de240-19b0-4362-97d3-f9f35ee7e3a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:25.475234 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16de240-19b0-4362-97d3-f9f35ee7e3a7 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:25.488734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1210/2407] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 14:00:25.522812 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3666e2d4-6123-4b23-87a0-0cde84ff3c3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '487fecd6467c4f0895411e60cf0a1612'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.524607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3666e2d4-6123-4b23-87a0-0cde84ff3c3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '487fecd6467c4f0895411e60cf0a1612'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.525338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3666e2d4-6123-4b23-87a0-0cde84ff3c3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '487fecd6467c4f0895411e60cf0a1612'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.543226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3666e2d4-6123-4b23-87a0-0cde84ff3c3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '487fecd6467c4f0895411e60cf0a1612'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.555316 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3666e2d4-6123-4b23-87a0-0cde84ff3c3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '487fecd6467c4f0895411e60cf0a1612'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:25.573294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1198/2408] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:25 2022] DELETE /identity/v3/users/487fecd6467c4f0895411e60cf0a1612 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:00:25.853394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16de240-19b0-4362-97d3-f9f35ee7e3a7 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:25.855265 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b16de240-19b0-4362-97d3-f9f35ee7e3a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:25.868169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16de240-19b0-4362-97d3-f9f35ee7e3a7 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:25.907213 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666e2d4-6123-4b23-87a0-0cde84ff3c3c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:25.908710 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3666e2d4-6123-4b23-87a0-0cde84ff3c3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:25.945036 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:25.945671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:25.945944 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:25.946211 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] PATH_INFO: `/v3/projects/62c7896dfbe44ea48236b9a6a63a9621` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:25.947361 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] RBAC: Authorizing `identity:delete_project(project_id=62c7896dfbe44ea48236b9a6a63a9621)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:25.949324 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:25.983137 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666e2d4-6123-4b23-87a0-0cde84ff3c3c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:25.987926 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:25.988569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:25.988845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:25.989114 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] PATH_INFO: `/v3/projects/b42adedff67b4a85896fee855d92d0b2` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:25.990209 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b42adedff67b4a85896fee855d92d0b2)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:25.992201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:25.998151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62c7896dfbe44ea48236b9a6a63a9621', 'request_id': 'req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.006358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62c7896dfbe44ea48236b9a6a63a9621', 'request_id': 'req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.013019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62c7896dfbe44ea48236b9a6a63a9621', 'request_id': 'req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.024635 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] The token cache is being invalidate because project 62c7896dfbe44ea48236b9a6a63a9621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:26.024922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.025554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.027268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1211/2409] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:25 2022] DELETE /identity/v3/projects/62c7896dfbe44ea48236b9a6a63a9621 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:26.031268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:26.032857 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:26.040853 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b42adedff67b4a85896fee855d92d0b2', 'request_id': 'req-3c58511f-a1af-4fe7-a859-37b335452872'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.047364 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b42adedff67b4a85896fee855d92d0b2', 'request_id': 'req-3c58511f-a1af-4fe7-a859-37b335452872'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.056626 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b42adedff67b4a85896fee855d92d0b2', 'request_id': 'req-3c58511f-a1af-4fe7-a859-37b335452872'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.063607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:26.068563 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] The token cache is being invalidate because project b42adedff67b4a85896fee855d92d0b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:26.069060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.071165 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.072881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1199/2410] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:25 2022] DELETE /identity/v3/projects/b42adedff67b4a85896fee855d92d0b2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:00:26.077344 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:26.079458 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:26.096948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:26.100773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e943e-d31a-44b7-9ea3-13a86be6a8be admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:26.147303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58511f-a1af-4fe7-a859-37b335452872 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:26.151933 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db1f21-a01e-4e8c-a773-b022976b757d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:26.152563 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db1f21-a01e-4e8c-a773-b022976b757d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:26.152834 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db1f21-a01e-4e8c-a773-b022976b757d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:26.153102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db1f21-a01e-4e8c-a773-b022976b757d admin admin] PATH_INFO: `/v3/users/a4b4205cb10946cea1c27b1c37ae8a57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:26.154512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db1f21-a01e-4e8c-a773-b022976b757d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4b4205cb10946cea1c27b1c37ae8a57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:26.156259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db1f21-a01e-4e8c-a773-b022976b757d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:26.159866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc960ab-ba56-4212-87d4-4081de9ce9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:26.160536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc960ab-ba56-4212-87d4-4081de9ce9e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:26.160821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc960ab-ba56-4212-87d4-4081de9ce9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:26.161088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc960ab-ba56-4212-87d4-4081de9ce9e4 admin admin] PATH_INFO: `/v3/users/451ddc5891d14be99787aedb541ccfe3` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:26.162232 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc960ab-ba56-4212-87d4-4081de9ce9e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=451ddc5891d14be99787aedb541ccfe3)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:26.164169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc960ab-ba56-4212-87d4-4081de9ce9e4 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:26.205189 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-69db1f21-a01e-4e8c-a773-b022976b757d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4b4205cb10946cea1c27b1c37ae8a57'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.205972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-69db1f21-a01e-4e8c-a773-b022976b757d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4b4205cb10946cea1c27b1c37ae8a57'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.206592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-69db1f21-a01e-4e8c-a773-b022976b757d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4b4205cb10946cea1c27b1c37ae8a57'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.213240 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-acc960ab-ba56-4212-87d4-4081de9ce9e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '451ddc5891d14be99787aedb541ccfe3'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.219658 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-69db1f21-a01e-4e8c-a773-b022976b757d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4b4205cb10946cea1c27b1c37ae8a57'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.220952 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-acc960ab-ba56-4212-87d4-4081de9ce9e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '451ddc5891d14be99787aedb541ccfe3'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.221821 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-acc960ab-ba56-4212-87d4-4081de9ce9e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '451ddc5891d14be99787aedb541ccfe3'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.222547 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-acc960ab-ba56-4212-87d4-4081de9ce9e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '451ddc5891d14be99787aedb541ccfe3'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.229455 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-69db1f21-a01e-4e8c-a773-b022976b757d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4b4205cb10946cea1c27b1c37ae8a57'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.233852 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-acc960ab-ba56-4212-87d4-4081de9ce9e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '451ddc5891d14be99787aedb541ccfe3'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.245450 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1200/2411] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:26 2022] DELETE /identity/v3/users/a4b4205cb10946cea1c27b1c37ae8a57 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:00:26.254586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1212/2412] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:26 2022] DELETE /identity/v3/users/451ddc5891d14be99787aedb541ccfe3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:26.536749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db1f21-a01e-4e8c-a773-b022976b757d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:26.537098 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc960ab-ba56-4212-87d4-4081de9ce9e4 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:26.538201 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-69db1f21-a01e-4e8c-a773-b022976b757d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:26.539090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-acc960ab-ba56-4212-87d4-4081de9ce9e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:26.615594 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc960ab-ba56-4212-87d4-4081de9ce9e4 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:26.620919 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:26.621784 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:26.622213 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:26.622514 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] PATH_INFO: `/v3/projects/eab0d4cc8bab479498c01211f66ea495` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:26.624884 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db1f21-a01e-4e8c-a773-b022976b757d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:26.625191 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab0d4cc8bab479498c01211f66ea495)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:26.627294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:26.632447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506cffe-9862-409f-92bd-a230b480150b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:26.633331 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506cffe-9862-409f-92bd-a230b480150b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:26.633778 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506cffe-9862-409f-92bd-a230b480150b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:26.634082 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506cffe-9862-409f-92bd-a230b480150b admin admin] PATH_INFO: `/v3/projects/61e0be896e5a49459cf7c2ffab05e19a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:26.635388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506cffe-9862-409f-92bd-a230b480150b admin admin] RBAC: Authorizing `identity:delete_project(project_id=61e0be896e5a49459cf7c2ffab05e19a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:26.637571 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506cffe-9862-409f-92bd-a230b480150b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:26.662497 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:26.671499 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:26.692397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab0d4cc8bab479498c01211f66ea495', 'request_id': 'req-a7b68093-3204-4d3b-8542-39808d79de48'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.698134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6506cffe-9862-409f-92bd-a230b480150b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61e0be896e5a49459cf7c2ffab05e19a', 'request_id': 'req-6506cffe-9862-409f-92bd-a230b480150b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.705025 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab0d4cc8bab479498c01211f66ea495', 'request_id': 'req-a7b68093-3204-4d3b-8542-39808d79de48'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.706519 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6506cffe-9862-409f-92bd-a230b480150b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61e0be896e5a49459cf7c2ffab05e19a', 'request_id': 'req-6506cffe-9862-409f-92bd-a230b480150b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.714739 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab0d4cc8bab479498c01211f66ea495', 'request_id': 'req-a7b68093-3204-4d3b-8542-39808d79de48'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.715297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6506cffe-9862-409f-92bd-a230b480150b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61e0be896e5a49459cf7c2ffab05e19a', 'request_id': 'req-6506cffe-9862-409f-92bd-a230b480150b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.728294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] The token cache is being invalidate because project eab0d4cc8bab479498c01211f66ea495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:26.728903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6506cffe-9862-409f-92bd-a230b480150b admin admin] The token cache is being invalidate because project 61e0be896e5a49459cf7c2ffab05e19a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:26.729017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.729132 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6506cffe-9862-409f-92bd-a230b480150b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.729693 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.730256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6506cffe-9862-409f-92bd-a230b480150b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.736242 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1213/2413] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:26 2022] DELETE /identity/v3/projects/eab0d4cc8bab479498c01211f66ea495 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:00:26.736413 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1201/2414] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:26 2022] DELETE /identity/v3/projects/61e0be896e5a49459cf7c2ffab05e19a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:00:26.737028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:26.737706 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bb8fa-4881-438b-b47c-7523167cbffe None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:26.737966 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bb8fa-4881-438b-b47c-7523167cbffe None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:26.738237 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bb8fa-4881-438b-b47c-7523167cbffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:26.738787 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:26.759741 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a00bb8fa-4881-438b-b47c-7523167cbffe None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:26.760979 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a00bb8fa-4881-438b-b47c-7523167cbffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:26.819979 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b68093-3204-4d3b-8542-39808d79de48 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:26.836454 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1202/2415] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 14:00:26.845902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d0ed4-f780-4f98-b06b-7f1c61df6fc8 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:26.846262 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d0ed4-f780-4f98-b06b-7f1c61df6fc8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:26.846601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d0ed4-f780-4f98-b06b-7f1c61df6fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:26.867719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-008d0ed4-f780-4f98-b06b-7f1c61df6fc8 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:26.868983 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-008d0ed4-f780-4f98-b06b-7f1c61df6fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:26.877648 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1203/2416] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 14:00:26.881471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f2d74-d0ad-4691-96f8-86a63fb9a71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:26.882055 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d0ed4-f780-4f98-b06b-7f1c61df6fc8 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:26.882166 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f2d74-d0ad-4691-96f8-86a63fb9a71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:26.882392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f2d74-d0ad-4691-96f8-86a63fb9a71e admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:26.882666 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f2d74-d0ad-4691-96f8-86a63fb9a71e admin admin] PATH_INFO: `/v3/users/53982ea779884c5692ad598f29269807` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:26.883936 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f2d74-d0ad-4691-96f8-86a63fb9a71e admin admin] RBAC: Authorizing `identity:delete_user(user_id=53982ea779884c5692ad598f29269807)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:26.889309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f2d74-d0ad-4691-96f8-86a63fb9a71e admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:26.892782 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d0ed4-f780-4f98-b06b-7f1c61df6fc8 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:26.897596 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b1f99-fc03-4120-972d-bd4a14131eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:26.898206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b1f99-fc03-4120-972d-bd4a14131eab admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:26.898476 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b1f99-fc03-4120-972d-bd4a14131eab admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:26.898738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b1f99-fc03-4120-972d-bd4a14131eab admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:26.899371 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b1f99-fc03-4120-972d-bd4a14131eab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:00:26.899687 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b1f99-fc03-4120-972d-bd4a14131eab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:26.901446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b1f99-fc03-4120-972d-bd4a14131eab admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:26.913716 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1204/2417] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 14:00:26.918852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b1f99-fc03-4120-972d-bd4a14131eab admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:26.929292 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b1f99-fc03-4120-972d-bd4a14131eab admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:26.934380 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54e812-8c67-4a0a-bbaf-d3ca2928a6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:26.935102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54e812-8c67-4a0a-bbaf-d3ca2928a6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:26.935386 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54e812-8c67-4a0a-bbaf-d3ca2928a6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:26.935859 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54e812-8c67-4a0a-bbaf-d3ca2928a6d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:26.936393 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54e812-8c67-4a0a-bbaf-d3ca2928a6d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:26.936967 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5a1f2d74-d0ad-4691-96f8-86a63fb9a71e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53982ea779884c5692ad598f29269807'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.938469 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54e812-8c67-4a0a-bbaf-d3ca2928a6d2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:26.943714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5a1f2d74-d0ad-4691-96f8-86a63fb9a71e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53982ea779884c5692ad598f29269807'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.944430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5a1f2d74-d0ad-4691-96f8-86a63fb9a71e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53982ea779884c5692ad598f29269807'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.945048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5a1f2d74-d0ad-4691-96f8-86a63fb9a71e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53982ea779884c5692ad598f29269807'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.953946 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-5a1f2d74-d0ad-4691-96f8-86a63fb9a71e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53982ea779884c5692ad598f29269807'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:26.955583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:26.972168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1205/2418] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:26 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 14:00:26.973271 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54e812-8c67-4a0a-bbaf-d3ca2928a6d2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:26.975232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8d54e812-8c67-4a0a-bbaf-d3ca2928a6d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:26.978989 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1214/2419] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:26 2022] DELETE /identity/v3/users/53982ea779884c5692ad598f29269807 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:27.040348 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54e812-8c67-4a0a-bbaf-d3ca2928a6d2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:27.050288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:27.103096 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1539ef-685d-4bca-ac56-0c4bdc0a3348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:27.104307 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1539ef-685d-4bca-ac56-0c4bdc0a3348 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.104587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1539ef-685d-4bca-ac56-0c4bdc0a3348 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.104857 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1539ef-685d-4bca-ac56-0c4bdc0a3348 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.105530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1539ef-685d-4bca-ac56-0c4bdc0a3348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:27.107951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1539ef-685d-4bca-ac56-0c4bdc0a3348 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:27.145647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1206/2420] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:26 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 14:00:27.150501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f2d74-d0ad-4691-96f8-86a63fb9a71e admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:27.166922 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f2d74-d0ad-4691-96f8-86a63fb9a71e admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:27.172512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eea629-9976-45d9-a045-001c51a07b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:27.173136 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eea629-9976-45d9-a045-001c51a07b30 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.173700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eea629-9976-45d9-a045-001c51a07b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.173700 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eea629-9976-45d9-a045-001c51a07b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.174355 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eea629-9976-45d9-a045-001c51a07b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:27.175992 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eea629-9976-45d9-a045-001c51a07b30 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:27.187489 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1215/2421] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:27.191703 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1539ef-685d-4bca-ac56-0c4bdc0a3348 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:27.202731 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1539ef-685d-4bca-ac56-0c4bdc0a3348 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:27.207529 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29200dab-0b18-46ff-ab03-c2e25195f0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:27.208311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29200dab-0b18-46ff-ab03-c2e25195f0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.208588 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29200dab-0b18-46ff-ab03-c2e25195f0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.208876 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29200dab-0b18-46ff-ab03-c2e25195f0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.209524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29200dab-0b18-46ff-ab03-c2e25195f0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:27.211138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29200dab-0b18-46ff-ab03-c2e25195f0d9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:27.221717 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1207/2422] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:27.226021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eea629-9976-45d9-a045-001c51a07b30 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:27.237276 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eea629-9976-45d9-a045-001c51a07b30 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:27.242169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5003e5-be58-4e50-bfff-4a464a56758b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:27.242888 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5003e5-be58-4e50-bfff-4a464a56758b admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.243223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5003e5-be58-4e50-bfff-4a464a56758b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.243495 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5003e5-be58-4e50-bfff-4a464a56758b admin admin] PATH_INFO: `/v3/projects/6e84c16ad7bd4c0d9398dbf8ee0e254e/users/550268002b2b4a0ca3ae1e5e36c76bbf/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.254606 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29200dab-0b18-46ff-ab03-c2e25195f0d9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:27.257565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-29200dab-0b18-46ff-ab03-c2e25195f0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:27.261294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5003e5-be58-4e50-bfff-4a464a56758b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e84c16ad7bd4c0d9398dbf8ee0e254e, user_id=550268002b2b4a0ca3ae1e5e36c76bbf, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:27.269377 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5003e5-be58-4e50-bfff-4a464a56758b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:27.274895 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29200dab-0b18-46ff-ab03-c2e25195f0d9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:27.281790 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:27.282407 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.282781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.282961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1 admin admin] PATH_INFO: `/v3/projects/e07a1c835d244b2286582627e0787be6` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.284107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e07a1c835d244b2286582627e0787be6)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:27.286032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:27.286564 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1216/2423] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:27 2022] PUT /identity/v3/projects/6e84c16ad7bd4c0d9398dbf8ee0e254e/users/550268002b2b4a0ca3ae1e5e36c76bbf/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:27.298111 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b3fe0-8036-496e-8f09-bd704410954a None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.298835 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b3fe0-8036-496e-8f09-bd704410954a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.298835 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b3fe0-8036-496e-8f09-bd704410954a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.328657 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-f04b3fe0-8036-496e-8f09-bd704410954a None None] MFA Rules not processed for user `550268002b2b4a0ca3ae1e5e36c76bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:27.330319 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f04b3fe0-8036-496e-8f09-bd704410954a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:27.337778 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e07a1c835d244b2286582627e0787be6', 'request_id': 'req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:27.345301 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e07a1c835d244b2286582627e0787be6', 'request_id': 'req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:27.355073 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e07a1c835d244b2286582627e0787be6', 'request_id': 'req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:27.366815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1 admin admin] The token cache is being invalidate because project e07a1c835d244b2286582627e0787be6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:27.367105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:27.367858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3d9de93f-87e2-4b1d-8a5d-780ffbf7bbb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:27.369721 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1208/2424] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:27 2022] DELETE /identity/v3/projects/e07a1c835d244b2286582627e0787be6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:00:27.377077 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49818542-6017-4008-8d4c-929942ee48d4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.377359 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49818542-6017-4008-8d4c-929942ee48d4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.377632 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49818542-6017-4008-8d4c-929942ee48d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.399805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1217/2425] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:27.400136 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-49818542-6017-4008-8d4c-929942ee48d4 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:27.401411 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-49818542-6017-4008-8d4c-929942ee48d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:27.411054 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b3fe0-8036-496e-8f09-bd704410954a None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:27.412553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f04b3fe0-8036-496e-8f09-bd704410954a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:27.469599 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1209/2426] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:27.475127 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b3fe0-8036-496e-8f09-bd704410954a None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:27.478159 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79f2b6-195a-4302-8980-2c3404aa1dba None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.478806 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79f2b6-195a-4302-8980-2c3404aa1dba None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.479076 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79f2b6-195a-4302-8980-2c3404aa1dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.479690 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca149f7-918e-41e6-97a2-5ed5b20027f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:27.480344 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca149f7-918e-41e6-97a2-5ed5b20027f0 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.480628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca149f7-918e-41e6-97a2-5ed5b20027f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.480891 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca149f7-918e-41e6-97a2-5ed5b20027f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.491343 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca149f7-918e-41e6-97a2-5ed5b20027f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:27.492741 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca149f7-918e-41e6-97a2-5ed5b20027f0 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:27.501120 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7a79f2b6-195a-4302-8980-2c3404aa1dba None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:27.502351 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7a79f2b6-195a-4302-8980-2c3404aa1dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:27.502749 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1218/2427] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 14:00:27.510960 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1210/2428] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 14:00:27.515430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca149f7-918e-41e6-97a2-5ed5b20027f0 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:27.525304 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79f2b6-195a-4302-8980-2c3404aa1dba None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:27.526786 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7a79f2b6-195a-4302-8980-2c3404aa1dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:27.527060 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca149f7-918e-41e6-97a2-5ed5b20027f0 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:27.532180 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611236ca-aa2a-4a0b-b988-e583039da8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:27.532779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611236ca-aa2a-4a0b-b988-e583039da8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.533055 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611236ca-aa2a-4a0b-b988-e583039da8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.533327 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611236ca-aa2a-4a0b-b988-e583039da8b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.534039 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611236ca-aa2a-4a0b-b988-e583039da8b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:00:27.534332 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611236ca-aa2a-4a0b-b988-e583039da8b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:27.536095 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611236ca-aa2a-4a0b-b988-e583039da8b3 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:27.539819 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79f2b6-195a-4302-8980-2c3404aa1dba None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:27.545152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a3dee-0108-4c99-842a-1f5ce1589150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:27.546045 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a3dee-0108-4c99-842a-1f5ce1589150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.546329 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a3dee-0108-4c99-842a-1f5ce1589150 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.546599 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a3dee-0108-4c99-842a-1f5ce1589150 admin admin] PATH_INFO: `/v3/users/550268002b2b4a0ca3ae1e5e36c76bbf` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.547755 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a3dee-0108-4c99-842a-1f5ce1589150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=550268002b2b4a0ca3ae1e5e36c76bbf)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:27.551308 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1219/2429] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 14:00:27.551448 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a3dee-0108-4c99-842a-1f5ce1589150 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:27.552364 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611236ca-aa2a-4a0b-b988-e583039da8b3 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:27.552770 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:27.563116 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611236ca-aa2a-4a0b-b988-e583039da8b3 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:27.569159 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1713e-1734-472c-aac9-5b98da5b8a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:27.569849 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1713e-1734-472c-aac9-5b98da5b8a17 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.570238 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1713e-1734-472c-aac9-5b98da5b8a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.570583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1713e-1734-472c-aac9-5b98da5b8a17 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.571027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1713e-1734-472c-aac9-5b98da5b8a17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:27.572982 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1713e-1734-472c-aac9-5b98da5b8a17 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:27.600751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4b1a3dee-0108-4c99-842a-1f5ce1589150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '550268002b2b4a0ca3ae1e5e36c76bbf'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:27.601538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4b1a3dee-0108-4c99-842a-1f5ce1589150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '550268002b2b4a0ca3ae1e5e36c76bbf'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:27.602312 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4b1a3dee-0108-4c99-842a-1f5ce1589150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '550268002b2b4a0ca3ae1e5e36c76bbf'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:27.603233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1220/2430] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:27 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:27.607977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1713e-1734-472c-aac9-5b98da5b8a17 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:27.609960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-edb1713e-1734-472c-aac9-5b98da5b8a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:27.615275 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:27.615682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4b1a3dee-0108-4c99-842a-1f5ce1589150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '550268002b2b4a0ca3ae1e5e36c76bbf'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:27.624793 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4b1a3dee-0108-4c99-842a-1f5ce1589150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '550268002b2b4a0ca3ae1e5e36c76bbf'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:27.638961 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1211/2431] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:27 2022] DELETE /identity/v3/users/550268002b2b4a0ca3ae1e5e36c76bbf => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:27.644258 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03557105-f1c7-49de-945b-8ccca735d83b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.644542 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03557105-f1c7-49de-945b-8ccca735d83b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.644815 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03557105-f1c7-49de-945b-8ccca735d83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.663968 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-03557105-f1c7-49de-945b-8ccca735d83b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:27.665162 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-03557105-f1c7-49de-945b-8ccca735d83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:27.679896 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1713e-1734-472c-aac9-5b98da5b8a17 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:27.729052 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1212/2432] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:27.739476 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03557105-f1c7-49de-945b-8ccca735d83b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:27.741152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-03557105-f1c7-49de-945b-8ccca735d83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:27.742456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2c2e1-4c65-41c5-a7e7-1eeaebef1fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:27.743004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2c2e1-4c65-41c5-a7e7-1eeaebef1fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.743278 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2c2e1-4c65-41c5-a7e7-1eeaebef1fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.743548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2c2e1-4c65-41c5-a7e7-1eeaebef1fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.744209 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2c2e1-4c65-41c5-a7e7-1eeaebef1fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:27.746102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2c2e1-4c65-41c5-a7e7-1eeaebef1fd8 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:27.775421 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1221/2433] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:27 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:27.778172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2c2e1-4c65-41c5-a7e7-1eeaebef1fd8 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:27.790692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2c2e1-4c65-41c5-a7e7-1eeaebef1fd8 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:27.797102 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d593d-8ce7-4b7f-8c1c-835208a75fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:27.798437 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d593d-8ce7-4b7f-8c1c-835208a75fbe admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.798437 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d593d-8ce7-4b7f-8c1c-835208a75fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.798437 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d593d-8ce7-4b7f-8c1c-835208a75fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.799103 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d593d-8ce7-4b7f-8c1c-835208a75fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:27.800942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d593d-8ce7-4b7f-8c1c-835208a75fbe admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:27.810208 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03557105-f1c7-49de-945b-8ccca735d83b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:27.815267 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1222/2434] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 14:00:27.815423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eceb8b1-936a-4749-b894-962da1e4fd1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:27.815423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceb8b1-936a-4749-b894-962da1e4fd1c service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.815423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceb8b1-936a-4749-b894-962da1e4fd1c service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.815716 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceb8b1-936a-4749-b894-962da1e4fd1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.818392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d593d-8ce7-4b7f-8c1c-835208a75fbe admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:27.826781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eceb8b1-936a-4749-b894-962da1e4fd1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:27.828819 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eceb8b1-936a-4749-b894-962da1e4fd1c service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:27.832225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d593d-8ce7-4b7f-8c1c-835208a75fbe admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:27.837944 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d0628-7be4-4365-ab47-be7aadedb167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:27.838604 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d0628-7be4-4365-ab47-be7aadedb167 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.838844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d0628-7be4-4365-ab47-be7aadedb167 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.839154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d0628-7be4-4365-ab47-be7aadedb167 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.839998 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d0628-7be4-4365-ab47-be7aadedb167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:27.841017 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1213/2435] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:27.841417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d0628-7be4-4365-ab47-be7aadedb167 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:27.856075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1223/2436] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:27.859022 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eceb8b1-936a-4749-b894-962da1e4fd1c service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:27.869006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eceb8b1-936a-4749-b894-962da1e4fd1c service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:27.879144 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac25017-5b62-409d-926f-8cda0cf21af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:27.879822 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac25017-5b62-409d-926f-8cda0cf21af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.880150 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac25017-5b62-409d-926f-8cda0cf21af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.880419 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac25017-5b62-409d-926f-8cda0cf21af3 admin admin] PATH_INFO: `/v3/projects/532c0bc5665e41b486e8bbb3f98d9a27/users/6d255337ed2b4c888537042717ed3119/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:27.898076 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac25017-5b62-409d-926f-8cda0cf21af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=532c0bc5665e41b486e8bbb3f98d9a27, user_id=6d255337ed2b4c888537042717ed3119, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:27.901001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac25017-5b62-409d-926f-8cda0cf21af3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:27.911538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:27.918049 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1214/2437] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:27 2022] PUT /identity/v3/projects/532c0bc5665e41b486e8bbb3f98d9a27/users/6d255337ed2b4c888537042717ed3119/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:27.994112 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1772c0-2ff5-48de-875f-15405bfeff95 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:27.994668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1772c0-2ff5-48de-875f-15405bfeff95 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:27.994797 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1772c0-2ff5-48de-875f-15405bfeff95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.029205 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-fa1772c0-2ff5-48de-875f-15405bfeff95 None None] MFA Rules not processed for user `6d255337ed2b4c888537042717ed3119`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:28.030747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-fa1772c0-2ff5-48de-875f-15405bfeff95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:28.104267 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1224/2438] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:28.108965 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac25017-5b62-409d-926f-8cda0cf21af3 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:28.176506 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac25017-5b62-409d-926f-8cda0cf21af3 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:28.181311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600cbd3-ed1e-46ca-a90d-c494ecd61991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:28.181906 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600cbd3-ed1e-46ca-a90d-c494ecd61991 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.182192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600cbd3-ed1e-46ca-a90d-c494ecd61991 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.182467 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600cbd3-ed1e-46ca-a90d-c494ecd61991 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.183102 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600cbd3-ed1e-46ca-a90d-c494ecd61991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:28.185033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600cbd3-ed1e-46ca-a90d-c494ecd61991 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:28.214393 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1215/2439] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:28 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:28.219242 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1772c0-2ff5-48de-875f-15405bfeff95 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:28.224300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600cbd3-ed1e-46ca-a90d-c494ecd61991 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:28.226757 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2600cbd3-ed1e-46ca-a90d-c494ecd61991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:28.232218 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1772c0-2ff5-48de-875f-15405bfeff95 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:28.236853 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:28.293725 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68219783-d2f5-4449-8dd9-c188e151365f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:28.294346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68219783-d2f5-4449-8dd9-c188e151365f admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.294629 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68219783-d2f5-4449-8dd9-c188e151365f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.294928 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68219783-d2f5-4449-8dd9-c188e151365f admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.295377 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600cbd3-ed1e-46ca-a90d-c494ecd61991 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:28.295561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68219783-d2f5-4449-8dd9-c188e151365f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:28.297898 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68219783-d2f5-4449-8dd9-c188e151365f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:28.300657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:28.301294 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.301566 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.301832 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9 admin admin] PATH_INFO: `/v3/projects/6e84c16ad7bd4c0d9398dbf8ee0e254e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.302982 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e84c16ad7bd4c0d9398dbf8ee0e254e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:28.304912 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:28.324629 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1225/2440] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:28 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:28.325714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68219783-d2f5-4449-8dd9-c188e151365f admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:28.335960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68219783-d2f5-4449-8dd9-c188e151365f admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:28.341101 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ad058-6397-4299-abe1-f84e36e37d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:28.341840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ad058-6397-4299-abe1-f84e36e37d7d admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.342110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ad058-6397-4299-abe1-f84e36e37d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.342382 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ad058-6397-4299-abe1-f84e36e37d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.343104 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ad058-6397-4299-abe1-f84e36e37d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:28.344765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ad058-6397-4299-abe1-f84e36e37d7d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:28.356178 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1226/2441] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:28.360472 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ad058-6397-4299-abe1-f84e36e37d7d admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:28.360857 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e84c16ad7bd4c0d9398dbf8ee0e254e', 'request_id': 'req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:28.367558 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e84c16ad7bd4c0d9398dbf8ee0e254e', 'request_id': 'req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:28.371411 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ad058-6397-4299-abe1-f84e36e37d7d admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:28.376760 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e84c16ad7bd4c0d9398dbf8ee0e254e', 'request_id': 'req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:28.389257 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9 admin admin] The token cache is being invalidate because project 6e84c16ad7bd4c0d9398dbf8ee0e254e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:28.389580 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:28.390250 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f1c8c0e1-e18e-4a62-b950-0db6afcbcfe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:28.392190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1216/2442] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:28 2022] DELETE /identity/v3/projects/6e84c16ad7bd4c0d9398dbf8ee0e254e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:00:28.397025 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbab92c-85b8-4f60-9cfb-ebe124c69885 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.397311 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbab92c-85b8-4f60-9cfb-ebe124c69885 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.397589 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbab92c-85b8-4f60-9cfb-ebe124c69885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.417414 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-adbab92c-85b8-4f60-9cfb-ebe124c69885 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:28.418628 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-adbab92c-85b8-4f60-9cfb-ebe124c69885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:28.428965 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:28.433451 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c78073-87b6-4ac0-9cd2-1272e10fa62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:28.434256 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c78073-87b6-4ac0-9cd2-1272e10fa62d admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.434541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c78073-87b6-4ac0-9cd2-1272e10fa62d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.434854 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c78073-87b6-4ac0-9cd2-1272e10fa62d admin admin] PATH_INFO: `/v3/projects/a7d00052e112447fb3903886636da0db/users/9c1ee07ccae84ea4a6d4bc092c65438b/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.450689 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c78073-87b6-4ac0-9cd2-1272e10fa62d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7d00052e112447fb3903886636da0db, user_id=9c1ee07ccae84ea4a6d4bc092c65438b, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:28.453511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c78073-87b6-4ac0-9cd2-1272e10fa62d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:28.468913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1227/2443] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:28 2022] PUT /identity/v3/projects/a7d00052e112447fb3903886636da0db/users/9c1ee07ccae84ea4a6d4bc092c65438b/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:00:28.473042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c78073-87b6-4ac0-9cd2-1272e10fa62d admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:28.474820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-d3c78073-87b6-4ac0-9cd2-1272e10fa62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:28.481209 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1217/2444] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 14:00:28.550843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2a38f-b841-4cbf-af33-d9cb34055b6c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.551120 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2a38f-b841-4cbf-af33-d9cb34055b6c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.551428 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2a38f-b841-4cbf-af33-d9cb34055b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.551624 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c78073-87b6-4ac0-9cd2-1272e10fa62d admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:28.557216 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de27ae-bcdf-4600-ad41-4ec3370130fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:28.557874 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de27ae-bcdf-4600-ad41-4ec3370130fa admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.558149 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de27ae-bcdf-4600-ad41-4ec3370130fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.558428 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de27ae-bcdf-4600-ad41-4ec3370130fa admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.559093 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de27ae-bcdf-4600-ad41-4ec3370130fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:28.560801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de27ae-bcdf-4600-ad41-4ec3370130fa admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:28.571630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b9d2a38f-b841-4cbf-af33-d9cb34055b6c None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:28.572169 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1228/2445] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:28.575980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de27ae-bcdf-4600-ad41-4ec3370130fa admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:28.576575 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b9d2a38f-b841-4cbf-af33-d9cb34055b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:28.582549 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:28.586763 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1218/2446] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 14:00:28.587408 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de27ae-bcdf-4600-ad41-4ec3370130fa admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:28.591013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2a38f-b841-4cbf-af33-d9cb34055b6c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:28.593643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483822e-9538-4418-9a90-34cfb4225d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:28.594796 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483822e-9538-4418-9a90-34cfb4225d66 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.595974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483822e-9538-4418-9a90-34cfb4225d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.595974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483822e-9538-4418-9a90-34cfb4225d66 admin admin] PATH_INFO: `/v3/projects/a7d00052e112447fb3903886636da0db/users/9c1ee07ccae84ea4a6d4bc092c65438b/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.598456 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483822e-9538-4418-9a90-34cfb4225d66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7d00052e112447fb3903886636da0db, user_id=9c1ee07ccae84ea4a6d4bc092c65438b, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:28.601263 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2a38f-b841-4cbf-af33-d9cb34055b6c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:28.601396 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483822e-9538-4418-9a90-34cfb4225d66 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:28.606646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8857b-f4c7-46ff-937d-1b3dff6d5e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:28.607287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8857b-f4c7-46ff-937d-1b3dff6d5e87 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.607627 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8857b-f4c7-46ff-937d-1b3dff6d5e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.608156 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8857b-f4c7-46ff-937d-1b3dff6d5e87 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.608709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8857b-f4c7-46ff-937d-1b3dff6d5e87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:00:28.609005 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8857b-f4c7-46ff-937d-1b3dff6d5e87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:28.610614 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8857b-f4c7-46ff-937d-1b3dff6d5e87 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:28.614634 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1229/2447] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:28 2022] PUT /identity/v3/projects/a7d00052e112447fb3903886636da0db/users/9c1ee07ccae84ea4a6d4bc092c65438b/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:28.623599 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1219/2448] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 14:00:28.625336 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c1a8b-c680-421a-bc16-e2020653ce6c None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.625727 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c1a8b-c680-421a-bc16-e2020653ce6c None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.626104 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c1a8b-c680-421a-bc16-e2020653ce6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.627780 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8857b-f4c7-46ff-937d-1b3dff6d5e87 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:28.642153 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8857b-f4c7-46ff-937d-1b3dff6d5e87 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:28.657623 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-bc8c1a8b-c680-421a-bc16-e2020653ce6c None None] MFA Rules not processed for user `9c1ee07ccae84ea4a6d4bc092c65438b`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:28.658781 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-bc8c1a8b-c680-421a-bc16-e2020653ce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:28.707711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2d3b4-cec0-472e-b168-65bf57c70e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:28.708537 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2d3b4-cec0-472e-b168-65bf57c70e17 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.708911 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2d3b4-cec0-472e-b168-65bf57c70e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.709275 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2d3b4-cec0-472e-b168-65bf57c70e17 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.710002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2d3b4-cec0-472e-b168-65bf57c70e17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:28.712086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2d3b4-cec0-472e-b168-65bf57c70e17 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:28.738313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1230/2449] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:28.743701 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1220/2450] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:28 2022] POST /identity/v3/projects => generated 361 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:28.748553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c1a8b-c680-421a-bc16-e2020653ce6c None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:28.753563 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2d3b4-cec0-472e-b168-65bf57c70e17 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:28.757403 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-21f2d3b4-cec0-472e-b168-65bf57c70e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:28.759681 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c1a8b-c680-421a-bc16-e2020653ce6c None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:28.829347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4879ca-1caa-4da5-a755-f00250c6ef83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:28.829873 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2d3b4-cec0-472e-b168-65bf57c70e17 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:28.830026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4879ca-1caa-4da5-a755-f00250c6ef83 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.830232 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4879ca-1caa-4da5-a755-f00250c6ef83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.830520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4879ca-1caa-4da5-a755-f00250c6ef83 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.831286 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4879ca-1caa-4da5-a755-f00250c6ef83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:28.833299 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4879ca-1caa-4da5-a755-f00250c6ef83 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:28.833478 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d247a5-b0df-4f52-8491-b8f132dcffd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:28.834089 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d247a5-b0df-4f52-8491-b8f132dcffd9 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.834365 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d247a5-b0df-4f52-8491-b8f132dcffd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.834666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d247a5-b0df-4f52-8491-b8f132dcffd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.836453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d9d247a5-b0df-4f52-8491-b8f132dcffd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:28.857250 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1231/2451] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:28 2022] POST /identity/v3/users => generated 281 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:28.861514 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4879ca-1caa-4da5-a755-f00250c6ef83 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:28.872161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4879ca-1caa-4da5-a755-f00250c6ef83 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:28.875837 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:28.877732 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e32a7-030a-4b31-8960-820865fe71db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:28.878358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e32a7-030a-4b31-8960-820865fe71db admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.878639 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e32a7-030a-4b31-8960-820865fe71db admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.878958 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e32a7-030a-4b31-8960-820865fe71db admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.879607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e32a7-030a-4b31-8960-820865fe71db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:28.881262 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e32a7-030a-4b31-8960-820865fe71db admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:28.892977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1232/2452] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 14:00:28.896972 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e32a7-030a-4b31-8960-820865fe71db admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:28.904715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d247a5-b0df-4f52-8491-b8f132dcffd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:28.906197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d247a5-b0df-4f52-8491-b8f132dcffd9 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:28.907132 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e32a7-030a-4b31-8960-820865fe71db admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:28.912440 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c35ea1-5ba7-479e-8bec-97da72f64812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:28.913081 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c35ea1-5ba7-479e-8bec-97da72f64812 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.913354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c35ea1-5ba7-479e-8bec-97da72f64812 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.913623 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c35ea1-5ba7-479e-8bec-97da72f64812 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.914411 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c35ea1-5ba7-479e-8bec-97da72f64812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:28.915636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1221/2453] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 14:00:28.916028 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c35ea1-5ba7-479e-8bec-97da72f64812 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:28.927921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1233/2454] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 14:00:28.932343 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d247a5-b0df-4f52-8491-b8f132dcffd9 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:28.943509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d247a5-b0df-4f52-8491-b8f132dcffd9 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:28.948289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b261f-b15c-43df-983a-91b94fbdb5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:28.948949 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b261f-b15c-43df-983a-91b94fbdb5da admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.949224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b261f-b15c-43df-983a-91b94fbdb5da admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.949488 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b261f-b15c-43df-983a-91b94fbdb5da admin admin] PATH_INFO: `/v3/projects/1f968bba76a14e599dbe7d859bf9772e/users/26a0f44195a043eeb5e0b8e1591b23ab/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:28.966354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b261f-b15c-43df-983a-91b94fbdb5da admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f968bba76a14e599dbe7d859bf9772e, user_id=26a0f44195a043eeb5e0b8e1591b23ab, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:28.969242 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b261f-b15c-43df-983a-91b94fbdb5da admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:28.987109 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1222/2455] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:28 2022] PUT /identity/v3/projects/1f968bba76a14e599dbe7d859bf9772e/users/26a0f44195a043eeb5e0b8e1591b23ab/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:00:28.997294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a8fdd-ee22-4b52-830c-c163f9d3da3a None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:28.997782 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a8fdd-ee22-4b52-830c-c163f9d3da3a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:28.997956 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a8fdd-ee22-4b52-830c-c163f9d3da3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:29.028128 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-3b4a8fdd-ee22-4b52-830c-c163f9d3da3a None None] MFA Rules not processed for user `26a0f44195a043eeb5e0b8e1591b23ab`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:29.029557 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3b4a8fdd-ee22-4b52-830c-c163f9d3da3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:29.069176 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:29.109607 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1234/2456] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:29.147921 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b261f-b15c-43df-983a-91b94fbdb5da admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:29.149851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-045b261f-b15c-43df-983a-91b94fbdb5da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:29.244121 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b261f-b15c-43df-983a-91b94fbdb5da admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:29.255113 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a9e2c-38a0-4a2c-949c-613da40a0de0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:29.256066 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a9e2c-38a0-4a2c-949c-613da40a0de0 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:29.257033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a9e2c-38a0-4a2c-949c-613da40a0de0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:29.258835 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a9e2c-38a0-4a2c-949c-613da40a0de0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:29.280789 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a9e2c-38a0-4a2c-949c-613da40a0de0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:29.282172 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a9e2c-38a0-4a2c-949c-613da40a0de0 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:29.297450 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1223/2457] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:29.313948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a8fdd-ee22-4b52-830c-c163f9d3da3a None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:29.384392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a8fdd-ee22-4b52-830c-c163f9d3da3a None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:29.389086 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e4575-af28-44a7-8078-40e1e93506c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:29.389840 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e4575-af28-44a7-8078-40e1e93506c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:29.390118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e4575-af28-44a7-8078-40e1e93506c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:29.390388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e4575-af28-44a7-8078-40e1e93506c0 admin admin] PATH_INFO: `/v3/users/26a0f44195a043eeb5e0b8e1591b23ab` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:29.391529 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e4575-af28-44a7-8078-40e1e93506c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26a0f44195a043eeb5e0b8e1591b23ab)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:29.393505 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e4575-af28-44a7-8078-40e1e93506c0 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:29.445513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-3e0e4575-af28-44a7-8078-40e1e93506c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26a0f44195a043eeb5e0b8e1591b23ab'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:29.452491 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-3e0e4575-af28-44a7-8078-40e1e93506c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26a0f44195a043eeb5e0b8e1591b23ab'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:29.453184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-3e0e4575-af28-44a7-8078-40e1e93506c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26a0f44195a043eeb5e0b8e1591b23ab'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:29.453789 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-3e0e4575-af28-44a7-8078-40e1e93506c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26a0f44195a043eeb5e0b8e1591b23ab'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:29.462275 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-3e0e4575-af28-44a7-8078-40e1e93506c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26a0f44195a043eeb5e0b8e1591b23ab'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:29.480811 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1235/2458] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:29 2022] DELETE /identity/v3/users/26a0f44195a043eeb5e0b8e1591b23ab => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:29.485670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b8f94-52de-46c1-9464-b1d78cb83b7f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:29.485975 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b8f94-52de-46c1-9464-b1d78cb83b7f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:29.486242 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b8f94-52de-46c1-9464-b1d78cb83b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:29.506124 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0e1b8f94-52de-46c1-9464-b1d78cb83b7f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:29.507370 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0e1b8f94-52de-46c1-9464-b1d78cb83b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:29.571772 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1224/2459] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:29.584135 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e4575-af28-44a7-8078-40e1e93506c0 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:29.585740 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3e0e4575-af28-44a7-8078-40e1e93506c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:29.648144 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e4575-af28-44a7-8078-40e1e93506c0 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:29.652753 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa080ba-b7a8-4b2a-b7d8-3ccbda2de009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:29.653367 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa080ba-b7a8-4b2a-b7d8-3ccbda2de009 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:29.653672 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa080ba-b7a8-4b2a-b7d8-3ccbda2de009 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:29.653939 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa080ba-b7a8-4b2a-b7d8-3ccbda2de009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:29.662479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:29.665346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa080ba-b7a8-4b2a-b7d8-3ccbda2de009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:29.667257 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa080ba-b7a8-4b2a-b7d8-3ccbda2de009 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:29.678939 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1236/2460] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:29.977158 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b8f94-52de-46c1-9464-b1d78cb83b7f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:29.979098 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0e1b8f94-52de-46c1-9464-b1d78cb83b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:30.003657 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b8f94-52de-46c1-9464-b1d78cb83b7f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:30.009297 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545f93a-fb3c-46ac-9b30-eae9fbad146a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:30.009941 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545f93a-fb3c-46ac-9b30-eae9fbad146a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:30.010210 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545f93a-fb3c-46ac-9b30-eae9fbad146a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:30.010474 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545f93a-fb3c-46ac-9b30-eae9fbad146a admin admin] PATH_INFO: `/v3/projects/1f968bba76a14e599dbe7d859bf9772e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:30.011669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545f93a-fb3c-46ac-9b30-eae9fbad146a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f968bba76a14e599dbe7d859bf9772e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:30.013759 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545f93a-fb3c-46ac-9b30-eae9fbad146a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:30.067181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5545f93a-fb3c-46ac-9b30-eae9fbad146a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f968bba76a14e599dbe7d859bf9772e', 'request_id': 'req-5545f93a-fb3c-46ac-9b30-eae9fbad146a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:30.068305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:30.076600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5545f93a-fb3c-46ac-9b30-eae9fbad146a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f968bba76a14e599dbe7d859bf9772e', 'request_id': 'req-5545f93a-fb3c-46ac-9b30-eae9fbad146a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:30.086958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5545f93a-fb3c-46ac-9b30-eae9fbad146a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f968bba76a14e599dbe7d859bf9772e', 'request_id': 'req-5545f93a-fb3c-46ac-9b30-eae9fbad146a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:30.098134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5545f93a-fb3c-46ac-9b30-eae9fbad146a admin admin] The token cache is being invalidate because project 1f968bba76a14e599dbe7d859bf9772e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:30.098418 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5545f93a-fb3c-46ac-9b30-eae9fbad146a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:30.099066 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5545f93a-fb3c-46ac-9b30-eae9fbad146a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:30.100642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1225/2461] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:29 2022] DELETE /identity/v3/projects/1f968bba76a14e599dbe7d859bf9772e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:30.106063 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944511e-c2ef-4029-a1c1-dc5e2b2d3702 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:30.106638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944511e-c2ef-4029-a1c1-dc5e2b2d3702 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:30.106638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944511e-c2ef-4029-a1c1-dc5e2b2d3702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:30.128301 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-8944511e-c2ef-4029-a1c1-dc5e2b2d3702 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:30.129509 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-8944511e-c2ef-4029-a1c1-dc5e2b2d3702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:30.192251 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1237/2462] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:30.200382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc4f24-c804-4666-9708-1c35e62b9d6c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:30.200682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc4f24-c804-4666-9708-1c35e62b9d6c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:30.201204 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc4f24-c804-4666-9708-1c35e62b9d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:30.222039 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7afc4f24-c804-4666-9708-1c35e62b9d6c None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:30.223235 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7afc4f24-c804-4666-9708-1c35e62b9d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:30.232868 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1226/2463] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:30.237627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944511e-c2ef-4029-a1c1-dc5e2b2d3702 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:30.247687 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944511e-c2ef-4029-a1c1-dc5e2b2d3702 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:30.253120 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa9136-9f3e-4195-a8c7-900099647d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:30.253734 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa9136-9f3e-4195-a8c7-900099647d8a admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:30.254012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa9136-9f3e-4195-a8c7-900099647d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:30.254282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa9136-9f3e-4195-a8c7-900099647d8a admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:30.254942 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa9136-9f3e-4195-a8c7-900099647d8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:00:30.255243 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa9136-9f3e-4195-a8c7-900099647d8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:30.256879 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa9136-9f3e-4195-a8c7-900099647d8a admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:30.269268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1238/2464] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 14:00:30.273840 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc4f24-c804-4666-9708-1c35e62b9d6c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:30.283415 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc4f24-c804-4666-9708-1c35e62b9d6c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:30.288263 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabf93e-174c-4ab5-9616-81396e22c470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:30.288872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabf93e-174c-4ab5-9616-81396e22c470 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:30.289143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabf93e-174c-4ab5-9616-81396e22c470 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:30.289409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabf93e-174c-4ab5-9616-81396e22c470 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:30.290173 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dabf93e-174c-4ab5-9616-81396e22c470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:30.292121 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dabf93e-174c-4ab5-9616-81396e22c470 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:30.334244 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1227/2465] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:00:30 2022] POST /identity/v3/projects => generated 363 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 14:00:30.339048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa9136-9f3e-4195-a8c7-900099647d8a admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:30.365309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa9136-9f3e-4195-a8c7-900099647d8a admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:30.432474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3a15a-ffda-43eb-90c3-d7f5ab745a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:30.433177 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3a15a-ffda-43eb-90c3-d7f5ab745a0f admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:30.433493 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3a15a-ffda-43eb-90c3-d7f5ab745a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:30.433815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3a15a-ffda-43eb-90c3-d7f5ab745a0f admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:30.434735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3a15a-ffda-43eb-90c3-d7f5ab745a0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:30.437417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3a15a-ffda-43eb-90c3-d7f5ab745a0f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:30.467909 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1239/2466] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:00:30 2022] POST /identity/v3/users => generated 282 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:00:30.469494 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabf93e-174c-4ab5-9616-81396e22c470 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:30.482414 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabf93e-174c-4ab5-9616-81396e22c470 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:30.487457 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502838b-6491-4f3b-a92d-b8e5ba375bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:30.488166 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502838b-6491-4f3b-a92d-b8e5ba375bab admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:30.488443 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502838b-6491-4f3b-a92d-b8e5ba375bab admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:30.488711 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502838b-6491-4f3b-a92d-b8e5ba375bab admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:30.489726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502838b-6491-4f3b-a92d-b8e5ba375bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:30.491001 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502838b-6491-4f3b-a92d-b8e5ba375bab admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:30.502510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1228/2467] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:30.506918 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3a15a-ffda-43eb-90c3-d7f5ab745a0f admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:30.517921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3a15a-ffda-43eb-90c3-d7f5ab745a0f admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:30.522691 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30931ec-ab8c-4327-bee6-3e87bcd4c686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:30.523310 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30931ec-ab8c-4327-bee6-3e87bcd4c686 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:30.523587 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30931ec-ab8c-4327-bee6-3e87bcd4c686 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:30.523857 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30931ec-ab8c-4327-bee6-3e87bcd4c686 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:30.524561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30931ec-ab8c-4327-bee6-3e87bcd4c686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:30.526196 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30931ec-ab8c-4327-bee6-3e87bcd4c686 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:30.537137 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1240/2468] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:00:30.541487 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502838b-6491-4f3b-a92d-b8e5ba375bab admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:30.551600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502838b-6491-4f3b-a92d-b8e5ba375bab admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:30.556290 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee555ef-3445-4239-84e3-d5e1953f51a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:30.556943 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee555ef-3445-4239-84e3-d5e1953f51a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:30.557219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee555ef-3445-4239-84e3-d5e1953f51a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:30.557489 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee555ef-3445-4239-84e3-d5e1953f51a3 admin admin] PATH_INFO: `/v3/projects/d778c0d63ca8402bae98a634814e4630/users/35af4b37089b4b16a0b438f41639c25b/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:30.572723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee555ef-3445-4239-84e3-d5e1953f51a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d778c0d63ca8402bae98a634814e4630, user_id=35af4b37089b4b16a0b438f41639c25b, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:30.575432 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee555ef-3445-4239-84e3-d5e1953f51a3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:30.591446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1229/2469] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:00:30 2022] PUT /identity/v3/projects/d778c0d63ca8402bae98a634814e4630/users/35af4b37089b4b16a0b438f41639c25b/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:30.604992 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59de58c-457d-41b4-b6d4-97b4d28a3800 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:30.605362 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59de58c-457d-41b4-b6d4-97b4d28a3800 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:30.605591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59de58c-457d-41b4-b6d4-97b4d28a3800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:30.638608 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-b59de58c-457d-41b4-b6d4-97b4d28a3800 None None] MFA Rules not processed for user `35af4b37089b4b16a0b438f41639c25b`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:30.640247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b59de58c-457d-41b4-b6d4-97b4d28a3800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:30.696569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:30.724804 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1241/2470] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:00:30.725252 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee555ef-3445-4239-84e3-d5e1953f51a3 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:30.727525 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-eee555ef-3445-4239-84e3-d5e1953f51a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:30.737073 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59de58c-457d-41b4-b6d4-97b4d28a3800 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:30.738533 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-b59de58c-457d-41b4-b6d4-97b4d28a3800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:30.803078 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59de58c-457d-41b4-b6d4-97b4d28a3800 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:30.806891 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ba9a7-87aa-4975-b9b3-84ada637f39f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:30.807513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ba9a7-87aa-4975-b9b3-84ada637f39f service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:30.807791 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ba9a7-87aa-4975-b9b3-84ada637f39f service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:30.808098 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ba9a7-87aa-4975-b9b3-84ada637f39f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:30.818368 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ba9a7-87aa-4975-b9b3-84ada637f39f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:30.819802 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ba9a7-87aa-4975-b9b3-84ada637f39f service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:30.824220 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee555ef-3445-4239-84e3-d5e1953f51a3 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:30.829523 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1242/2471] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:30.830502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13b314-7d5f-4d97-ac60-16fec28ade3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:30.831288 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13b314-7d5f-4d97-ac60-16fec28ade3d service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:30.831572 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13b314-7d5f-4d97-ac60-16fec28ade3d service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:30.831844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13b314-7d5f-4d97-ac60-16fec28ade3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:30.833799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9b13b314-7d5f-4d97-ac60-16fec28ade3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:30.849241 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13b314-7d5f-4d97-ac60-16fec28ade3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:30.850850 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13b314-7d5f-4d97-ac60-16fec28ade3d service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:30.862303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1230/2472] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 14:00:31.550844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ba9a7-87aa-4975-b9b3-84ada637f39f service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:31.552473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c97ba9a7-87aa-4975-b9b3-84ada637f39f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:31.637574 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ba9a7-87aa-4975-b9b3-84ada637f39f service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:31.643057 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157782f-63e0-4067-a262-5eb967469087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:31.643707 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157782f-63e0-4067-a262-5eb967469087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:31.644014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157782f-63e0-4067-a262-5eb967469087 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:31.644282 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157782f-63e0-4067-a262-5eb967469087 admin admin] PATH_INFO: `/v3/users/6d255337ed2b4c888537042717ed3119` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:31.645422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157782f-63e0-4067-a262-5eb967469087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d255337ed2b4c888537042717ed3119)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:31.647349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157782f-63e0-4067-a262-5eb967469087 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:31.696533 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-1157782f-63e0-4067-a262-5eb967469087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d255337ed2b4c888537042717ed3119'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:31.704255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-1157782f-63e0-4067-a262-5eb967469087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d255337ed2b4c888537042717ed3119'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:31.705008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-1157782f-63e0-4067-a262-5eb967469087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d255337ed2b4c888537042717ed3119'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:31.705722 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-1157782f-63e0-4067-a262-5eb967469087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d255337ed2b4c888537042717ed3119'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:31.716154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-1157782f-63e0-4067-a262-5eb967469087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d255337ed2b4c888537042717ed3119'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:31.736124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1243/2473] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:31 2022] DELETE /identity/v3/users/6d255337ed2b4c888537042717ed3119 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:31.740738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdae35-d996-498c-9a92-ff3d4504268c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:31.741253 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdae35-d996-498c-9a92-ff3d4504268c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:31.741253 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdae35-d996-498c-9a92-ff3d4504268c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:31.760301 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5bcdae35-d996-498c-9a92-ff3d4504268c None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:00:31.761385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5bcdae35-d996-498c-9a92-ff3d4504268c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:31.815898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:31.825623 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1231/2474] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 14:00:31.835830 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157782f-63e0-4067-a262-5eb967469087 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:31.837422 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-1157782f-63e0-4067-a262-5eb967469087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:31.853762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:31.906981 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157782f-63e0-4067-a262-5eb967469087 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:31.910814 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf6b33-a21f-4d32-9347-723e000349ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:31.911433 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf6b33-a21f-4d32-9347-723e000349ed service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:31.911716 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf6b33-a21f-4d32-9347-723e000349ed service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:31.912002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf6b33-a21f-4d32-9347-723e000349ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:31.921142 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf6b33-a21f-4d32-9347-723e000349ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:31.922554 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf6b33-a21f-4d32-9347-723e000349ed service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:31.932587 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1244/2475] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:32.238063 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdae35-d996-498c-9a92-ff3d4504268c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:32.239649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5bcdae35-d996-498c-9a92-ff3d4504268c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:32.254565 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdae35-d996-498c-9a92-ff3d4504268c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:32.259373 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:32.260484 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:32.260484 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:32.260700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] PATH_INFO: `/v3/projects/532c0bc5665e41b486e8bbb3f98d9a27` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:32.261972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] RBAC: Authorizing `identity:delete_project(project_id=532c0bc5665e41b486e8bbb3f98d9a27)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:32.263997 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:32.320276 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '532c0bc5665e41b486e8bbb3f98d9a27', 'request_id': 'req-4d2bd065-d1e4-414d-8b84-62c636fa954c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:32.327144 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '532c0bc5665e41b486e8bbb3f98d9a27', 'request_id': 'req-4d2bd065-d1e4-414d-8b84-62c636fa954c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:32.336291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '532c0bc5665e41b486e8bbb3f98d9a27', 'request_id': 'req-4d2bd065-d1e4-414d-8b84-62c636fa954c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:32.350326 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] The token cache is being invalidate because project 532c0bc5665e41b486e8bbb3f98d9a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:32.350614 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:32.351267 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:32.352836 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1232/2476] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:32 2022] DELETE /identity/v3/projects/532c0bc5665e41b486e8bbb3f98d9a27 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:32.357428 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf6b33-a21f-4d32-9347-723e000349ed service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:32.367300 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf6b33-a21f-4d32-9347-723e000349ed service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:32.427731 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9bf1f6-b299-4668-9712-a22fa8018427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:32.428501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bf1f6-b299-4668-9712-a22fa8018427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:32.428902 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bf1f6-b299-4668-9712-a22fa8018427 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:32.429256 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9bf1f6-b299-4668-9712-a22fa8018427 admin admin] PATH_INFO: `/v3/users/9c1ee07ccae84ea4a6d4bc092c65438b` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:32.430474 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9bf1f6-b299-4668-9712-a22fa8018427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c1ee07ccae84ea4a6d4bc092c65438b)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:32.432584 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9bf1f6-b299-4668-9712-a22fa8018427 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:32.493310 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9e9bf1f6-b299-4668-9712-a22fa8018427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c1ee07ccae84ea4a6d4bc092c65438b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:32.501458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9e9bf1f6-b299-4668-9712-a22fa8018427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c1ee07ccae84ea4a6d4bc092c65438b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:32.502227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9e9bf1f6-b299-4668-9712-a22fa8018427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c1ee07ccae84ea4a6d4bc092c65438b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:32.502926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9e9bf1f6-b299-4668-9712-a22fa8018427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c1ee07ccae84ea4a6d4bc092c65438b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:32.512795 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-9e9bf1f6-b299-4668-9712-a22fa8018427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c1ee07ccae84ea4a6d4bc092c65438b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:32.537774 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1245/2477] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:00:32 2022] DELETE /identity/v3/users/9c1ee07ccae84ea4a6d4bc092c65438b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:32.952546 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:32.954031 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:32.979239 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bd065-d1e4-414d-8b84-62c636fa954c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:33.040946 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:33.041771 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:33.042054 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:33.042315 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] PATH_INFO: `/v3/projects/a7d00052e112447fb3903886636da0db` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:33.043502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7d00052e112447fb3903886636da0db)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:33.045562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:33.094029 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7d00052e112447fb3903886636da0db', 'request_id': 'req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:33.100851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7d00052e112447fb3903886636da0db', 'request_id': 'req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:33.112300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7d00052e112447fb3903886636da0db', 'request_id': 'req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:33.124569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] The token cache is being invalidate because project a7d00052e112447fb3903886636da0db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:00:33.124918 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:33.125584 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:00:33.127090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1233/2478] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:00:32 2022] DELETE /identity/v3/projects/a7d00052e112447fb3903886636da0db => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:00:33.148104 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9bf1f6-b299-4668-9712-a22fa8018427 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:33.149658 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-9e9bf1f6-b299-4668-9712-a22fa8018427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:33.194964 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:33.217651 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9bf1f6-b299-4668-9712-a22fa8018427 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:33.283775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414db305-1dfa-4378-87e9-924d4996cbbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:33.284450 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414db305-1dfa-4378-87e9-924d4996cbbd service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:33.284728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414db305-1dfa-4378-87e9-924d4996cbbd service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:33.284997 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414db305-1dfa-4378-87e9-924d4996cbbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:33.286907 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-414db305-1dfa-4378-87e9-924d4996cbbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:33.358656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414db305-1dfa-4378-87e9-924d4996cbbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:33.360157 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414db305-1dfa-4378-87e9-924d4996cbbd service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:33.371337 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1246/2479] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:35.105515 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:00:35.106714 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:35.130496 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:35.225686 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec56651-76e9-4d9d-9139-6d6e6e9f6845 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:00:35.234819 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f25be-ec86-4cbb-a4a5-cfd32c01b482 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:00:35.235454 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f25be-ec86-4cbb-a4a5-cfd32c01b482 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:00:35.235733 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f25be-ec86-4cbb-a4a5-cfd32c01b482 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:00:35.236040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f25be-ec86-4cbb-a4a5-cfd32c01b482 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:00:35.237902 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-af8f25be-ec86-4cbb-a4a5-cfd32c01b482 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:00:35.254496 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f25be-ec86-4cbb-a4a5-cfd32c01b482 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:00:35.256157 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f25be-ec86-4cbb-a4a5-cfd32c01b482 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:00:35.274899 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1234/2480] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:00:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:00:43.366334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=54 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:00:45.269323 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=28 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:01:15.035943 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414db305-1dfa-4378-87e9-924d4996cbbd service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:01:15.037661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-414db305-1dfa-4378-87e9-924d4996cbbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:01:15.119019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414db305-1dfa-4378-87e9-924d4996cbbd service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:01:15.124283 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f4c59-58f7-42a1-a53a-94795a9c7c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:01:15.124927 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f4c59-58f7-42a1-a53a-94795a9c7c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:15.125200 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f4c59-58f7-42a1-a53a-94795a9c7c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:15.125465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f4c59-58f7-42a1-a53a-94795a9c7c35 admin admin] PATH_INFO: `/v3/users/35af4b37089b4b16a0b438f41639c25b` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:15.126684 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f4c59-58f7-42a1-a53a-94795a9c7c35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35af4b37089b4b16a0b438f41639c25b)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:01:15.128697 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f4c59-58f7-42a1-a53a-94795a9c7c35 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:01:15.177242 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-508f4c59-58f7-42a1-a53a-94795a9c7c35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35af4b37089b4b16a0b438f41639c25b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:15.184205 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-508f4c59-58f7-42a1-a53a-94795a9c7c35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35af4b37089b4b16a0b438f41639c25b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:15.185053 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-508f4c59-58f7-42a1-a53a-94795a9c7c35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35af4b37089b4b16a0b438f41639c25b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:15.185741 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-508f4c59-58f7-42a1-a53a-94795a9c7c35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35af4b37089b4b16a0b438f41639c25b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:15.193977 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-508f4c59-58f7-42a1-a53a-94795a9c7c35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35af4b37089b4b16a0b438f41639c25b'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:15.212577 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1247/2481] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:01:15 2022] DELETE /identity/v3/users/35af4b37089b4b16a0b438f41639c25b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:01:15.217152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290d247-60d6-40e1-ab3f-d74ce64aaa4f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:15.217718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290d247-60d6-40e1-ab3f-d74ce64aaa4f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:15.217718 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290d247-60d6-40e1-ab3f-d74ce64aaa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:15.238569 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6290d247-60d6-40e1-ab3f-d74ce64aaa4f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:01:15.239786 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6290d247-60d6-40e1-ab3f-d74ce64aaa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:01:15.303672 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1235/2482] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:01:15.314007 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f4c59-58f7-42a1-a53a-94795a9c7c35 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:01:15.315548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-508f4c59-58f7-42a1-a53a-94795a9c7c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:01:15.381834 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f4c59-58f7-42a1-a53a-94795a9c7c35 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:01:15.386346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ac4b6-eb39-4c5c-ac13-75e1849b3e86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:01:15.386999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ac4b6-eb39-4c5c-ac13-75e1849b3e86 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:15.387299 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ac4b6-eb39-4c5c-ac13-75e1849b3e86 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:15.387572 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ac4b6-eb39-4c5c-ac13-75e1849b3e86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:15.398193 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ac4b6-eb39-4c5c-ac13-75e1849b3e86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:01:15.400007 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ac4b6-eb39-4c5c-ac13-75e1849b3e86 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:01:15.410773 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1248/2483] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:01:15.448133 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98debb8-5d32-4146-880c-e5e8cf1a75ba None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:15.448505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98debb8-5d32-4146-880c-e5e8cf1a75ba None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:15.448843 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98debb8-5d32-4146-880c-e5e8cf1a75ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:15.468860 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b98debb8-5d32-4146-880c-e5e8cf1a75ba None None] MFA Rules not processed for user `a10fc1a48c9c41eaa1bf3ff2e4109657`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:01:15.470086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b98debb8-5d32-4146-880c-e5e8cf1a75ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:01:15.531207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1236/2484] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:01:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:01:15.705354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ac4b6-eb39-4c5c-ac13-75e1849b3e86 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:01:15.706996 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-955ac4b6-eb39-4c5c-ac13-75e1849b3e86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:01:15.722297 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ac4b6-eb39-4c5c-ac13-75e1849b3e86 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:01:15.728425 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca05fa-9c91-4042-aec9-685a522582be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:01:15.729107 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca05fa-9c91-4042-aec9-685a522582be admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:15.729988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca05fa-9c91-4042-aec9-685a522582be admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:15.729988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca05fa-9c91-4042-aec9-685a522582be admin admin] PATH_INFO: `/v3/projects/d778c0d63ca8402bae98a634814e4630` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:15.730838 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca05fa-9c91-4042-aec9-685a522582be admin admin] RBAC: Authorizing `identity:delete_project(project_id=d778c0d63ca8402bae98a634814e4630)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:01:15.732934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca05fa-9c91-4042-aec9-685a522582be admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:01:15.751088 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:01:15.783204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-83ca05fa-9c91-4042-aec9-685a522582be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd778c0d63ca8402bae98a634814e4630', 'request_id': 'req-83ca05fa-9c91-4042-aec9-685a522582be'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:15.792224 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-83ca05fa-9c91-4042-aec9-685a522582be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd778c0d63ca8402bae98a634814e4630', 'request_id': 'req-83ca05fa-9c91-4042-aec9-685a522582be'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:15.801438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-83ca05fa-9c91-4042-aec9-685a522582be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd778c0d63ca8402bae98a634814e4630', 'request_id': 'req-83ca05fa-9c91-4042-aec9-685a522582be'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:15.815022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-83ca05fa-9c91-4042-aec9-685a522582be admin admin] The token cache is being invalidate because project d778c0d63ca8402bae98a634814e4630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:01:15.815295 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-83ca05fa-9c91-4042-aec9-685a522582be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:15.815994 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-83ca05fa-9c91-4042-aec9-685a522582be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:15.817528 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1249/2485] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:01:15 2022] DELETE /identity/v3/projects/d778c0d63ca8402bae98a634814e4630 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:01:15.822670 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d201d6-ea4b-433e-8d0c-6dd86db8dc3f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:15.823023 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d201d6-ea4b-433e-8d0c-6dd86db8dc3f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:15.823385 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d201d6-ea4b-433e-8d0c-6dd86db8dc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:15.843225 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-30d201d6-ea4b-433e-8d0c-6dd86db8dc3f None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:01:15.844458 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-30d201d6-ea4b-433e-8d0c-6dd86db8dc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:01:15.912388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1237/2486] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 14:01:15.920396 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e230571-ee32-4312-9386-97e8d687a4a5 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:15.920680 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e230571-ee32-4312-9386-97e8d687a4a5 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:15.920955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e230571-ee32-4312-9386-97e8d687a4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:15.941359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-2e230571-ee32-4312-9386-97e8d687a4a5 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:01:15.942583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2e230571-ee32-4312-9386-97e8d687a4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:01:15.950670 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1250/2487] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:01:15.955080 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d201d6-ea4b-433e-8d0c-6dd86db8dc3f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:01:15.965717 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d201d6-ea4b-433e-8d0c-6dd86db8dc3f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:01:15.970727 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ca72d-b8ff-44c0-920a-48cf63708c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:01:15.971357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ca72d-b8ff-44c0-920a-48cf63708c2c admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:15.971630 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ca72d-b8ff-44c0-920a-48cf63708c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:15.971896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ca72d-b8ff-44c0-920a-48cf63708c2c admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:15.972564 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ca72d-b8ff-44c0-920a-48cf63708c2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:01:15.972848 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ca72d-b8ff-44c0-920a-48cf63708c2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:01:15.974482 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ca72d-b8ff-44c0-920a-48cf63708c2c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:01:15.987938 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1238/2488] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 14:01:15.992403 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e230571-ee32-4312-9386-97e8d687a4a5 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:01:16.002330 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e230571-ee32-4312-9386-97e8d687a4a5 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:01:16.007471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ec653-8851-4006-a50d-7069290f9ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:01:16.008272 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ec653-8851-4006-a50d-7069290f9ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:16.008500 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ec653-8851-4006-a50d-7069290f9ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:16.008671 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ec653-8851-4006-a50d-7069290f9ce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:16.009492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ec653-8851-4006-a50d-7069290f9ce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:01:16.011434 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ec653-8851-4006-a50d-7069290f9ce3 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:01:16.042110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1251/2489] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:01:15 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:01:16.046538 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ca72d-b8ff-44c0-920a-48cf63708c2c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:01:16.058072 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ca72d-b8ff-44c0-920a-48cf63708c2c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:01:16.089653 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:01:16.119749 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d97adc-148c-41ea-9c67-c773cd3cff8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:01:16.120404 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d97adc-148c-41ea-9c67-c773cd3cff8a admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:16.120679 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d97adc-148c-41ea-9c67-c773cd3cff8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:16.121080 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d97adc-148c-41ea-9c67-c773cd3cff8a admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:16.121715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d97adc-148c-41ea-9c67-c773cd3cff8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:01:16.123650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d97adc-148c-41ea-9c67-c773cd3cff8a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:01:16.149402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1239/2490] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:01:16 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:01:16.150911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ec653-8851-4006-a50d-7069290f9ce3 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:01:16.162093 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ec653-8851-4006-a50d-7069290f9ce3 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:01:16.168948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85335ea1-6086-4515-8273-beade23290cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:01:16.169886 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85335ea1-6086-4515-8273-beade23290cc admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:16.170233 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85335ea1-6086-4515-8273-beade23290cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:16.170571 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85335ea1-6086-4515-8273-beade23290cc admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:16.171291 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85335ea1-6086-4515-8273-beade23290cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:01:16.173100 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85335ea1-6086-4515-8273-beade23290cc admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:01:16.187925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1252/2491] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:01:16.189936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d97adc-148c-41ea-9c67-c773cd3cff8a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:01:16.201773 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d97adc-148c-41ea-9c67-c773cd3cff8a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:01:16.207014 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452de3f-d873-4eca-ac75-a7fd9d473036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:01:16.207714 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452de3f-d873-4eca-ac75-a7fd9d473036 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:16.208090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452de3f-d873-4eca-ac75-a7fd9d473036 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:16.208445 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452de3f-d873-4eca-ac75-a7fd9d473036 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:16.209090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452de3f-d873-4eca-ac75-a7fd9d473036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:01:16.210708 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452de3f-d873-4eca-ac75-a7fd9d473036 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:01:16.223933 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1240/2492] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:01:16.226928 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85335ea1-6086-4515-8273-beade23290cc admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:01:16.239134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85335ea1-6086-4515-8273-beade23290cc admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:01:16.248451 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a54d6-916d-4d26-9675-a192510dfcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:01:16.249138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a54d6-916d-4d26-9675-a192510dfcef admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:16.249468 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a54d6-916d-4d26-9675-a192510dfcef admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:16.249742 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a54d6-916d-4d26-9675-a192510dfcef admin admin] PATH_INFO: `/v3/projects/8d10b37dd2b24e6fa5e0a5800b5a1d80/users/ea78d95f9ca044579105672e02ce4852/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:16.265779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a54d6-916d-4d26-9675-a192510dfcef admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d10b37dd2b24e6fa5e0a5800b5a1d80, user_id=ea78d95f9ca044579105672e02ce4852, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:01:16.268479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a54d6-916d-4d26-9675-a192510dfcef admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:01:16.285386 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1253/2493] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:01:16 2022] PUT /identity/v3/projects/8d10b37dd2b24e6fa5e0a5800b5a1d80/users/ea78d95f9ca044579105672e02ce4852/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:01:16.297612 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91ce95-3a25-4a5a-a2de-316243e3fd2a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:16.298540 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91ce95-3a25-4a5a-a2de-316243e3fd2a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:16.298918 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91ce95-3a25-4a5a-a2de-316243e3fd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:16.329662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5d91ce95-3a25-4a5a-a2de-316243e3fd2a None None] MFA Rules not processed for user `ea78d95f9ca044579105672e02ce4852`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:01:16.330939 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5d91ce95-3a25-4a5a-a2de-316243e3fd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:01:16.397765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1241/2494] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:01:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 15 14:01:16.411117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a54d6-916d-4d26-9675-a192510dfcef admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:01:16.412913 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-694a54d6-916d-4d26-9675-a192510dfcef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:01:16.473080 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a54d6-916d-4d26-9675-a192510dfcef admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:01:16.478027 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ab464-b670-4562-b5f4-45b3e4d8778c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:01:16.478653 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ab464-b670-4562-b5f4-45b3e4d8778c service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:16.478932 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ab464-b670-4562-b5f4-45b3e4d8778c service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:16.479205 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ab464-b670-4562-b5f4-45b3e4d8778c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:16.488657 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ab464-b670-4562-b5f4-45b3e4d8778c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:01:16.490110 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ab464-b670-4562-b5f4-45b3e4d8778c service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:01:16.499933 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1254/2495] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:01:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:01:16.527740 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91ce95-3a25-4a5a-a2de-316243e3fd2a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:01:16.599500 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91ce95-3a25-4a5a-a2de-316243e3fd2a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:01:16.605400 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242bc74-2bc1-4dc6-b0e4-ded659ab5158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:01:16.606285 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242bc74-2bc1-4dc6-b0e4-ded659ab5158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:16.606709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242bc74-2bc1-4dc6-b0e4-ded659ab5158 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:16.606963 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242bc74-2bc1-4dc6-b0e4-ded659ab5158 admin admin] PATH_INFO: `/v3/users/ea78d95f9ca044579105672e02ce4852` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:16.608206 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242bc74-2bc1-4dc6-b0e4-ded659ab5158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea78d95f9ca044579105672e02ce4852)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:01:16.611496 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242bc74-2bc1-4dc6-b0e4-ded659ab5158 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:01:16.667202 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:01:16.670805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2242bc74-2bc1-4dc6-b0e4-ded659ab5158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea78d95f9ca044579105672e02ce4852'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:16.671645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2242bc74-2bc1-4dc6-b0e4-ded659ab5158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea78d95f9ca044579105672e02ce4852'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:16.672232 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2242bc74-2bc1-4dc6-b0e4-ded659ab5158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea78d95f9ca044579105672e02ce4852'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:16.688151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2242bc74-2bc1-4dc6-b0e4-ded659ab5158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea78d95f9ca044579105672e02ce4852'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:16.697457 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2242bc74-2bc1-4dc6-b0e4-ded659ab5158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea78d95f9ca044579105672e02ce4852'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:16.713750 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1242/2496] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:01:16 2022] DELETE /identity/v3/users/ea78d95f9ca044579105672e02ce4852 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:01:16.718460 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de9cc3-23c5-4fc6-8c52-a3fef42b82a3 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:16.718747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de9cc3-23c5-4fc6-8c52-a3fef42b82a3 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:16.719026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de9cc3-23c5-4fc6-8c52-a3fef42b82a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:16.739679 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-e4de9cc3-23c5-4fc6-8c52-a3fef42b82a3 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:01:16.740966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e4de9cc3-23c5-4fc6-8c52-a3fef42b82a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:01:16.783526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:01:16.805962 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1255/2497] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:01:16.816811 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242bc74-2bc1-4dc6-b0e4-ded659ab5158 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:01:16.818353 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2242bc74-2bc1-4dc6-b0e4-ded659ab5158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:01:16.882490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242bc74-2bc1-4dc6-b0e4-ded659ab5158 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:01:16.886890 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0747049-8ecf-4f46-9890-9a89cfb6f898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:01:16.887647 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0747049-8ecf-4f46-9890-9a89cfb6f898 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:16.888047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0747049-8ecf-4f46-9890-9a89cfb6f898 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:16.888319 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0747049-8ecf-4f46-9890-9a89cfb6f898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:16.898896 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0747049-8ecf-4f46-9890-9a89cfb6f898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:01:16.900449 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0747049-8ecf-4f46-9890-9a89cfb6f898 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:01:16.910629 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1243/2498] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:01:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:01:17.197497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de9cc3-23c5-4fc6-8c52-a3fef42b82a3 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:01:17.199122 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-e4de9cc3-23c5-4fc6-8c52-a3fef42b82a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:01:17.228798 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de9cc3-23c5-4fc6-8c52-a3fef42b82a3 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:01:17.235541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:01:17.242083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:17.242083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:17.242083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] PATH_INFO: `/v3/projects/8d10b37dd2b24e6fa5e0a5800b5a1d80` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:17.244155 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d10b37dd2b24e6fa5e0a5800b5a1d80)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:01:17.246726 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:01:17.311319 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d10b37dd2b24e6fa5e0a5800b5a1d80', 'request_id': 'req-fc306147-8d61-4673-aa5d-da1dc9ca7196'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:17.321347 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d10b37dd2b24e6fa5e0a5800b5a1d80', 'request_id': 'req-fc306147-8d61-4673-aa5d-da1dc9ca7196'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:17.329522 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d10b37dd2b24e6fa5e0a5800b5a1d80', 'request_id': 'req-fc306147-8d61-4673-aa5d-da1dc9ca7196'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:17.348045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] The token cache is being invalidate because project 8d10b37dd2b24e6fa5e0a5800b5a1d80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:01:17.348574 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:17.349487 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:01:17.352961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1256/2499] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:01:17 2022] DELETE /identity/v3/projects/8d10b37dd2b24e6fa5e0a5800b5a1d80 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:01:26.906274 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:DELETE=4,keystone:SELECT=39,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:01:27.343741 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:01:52.673663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0747049-8ecf-4f46-9890-9a89cfb6f898 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:01:52.675404 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e0747049-8ecf-4f46-9890-9a89cfb6f898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:01:52.750185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0747049-8ecf-4f46-9890-9a89cfb6f898 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:01:52.815053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb684d-7b77-44b5-8329-7527d0c66101 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6720518c03413aa8f85e4d6142fe87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:01:52.815673 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb684d-7b77-44b5-8329-7527d0c66101 service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:01:52.815988 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb684d-7b77-44b5-8329-7527d0c66101 service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:01:52.816259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb684d-7b77-44b5-8329-7527d0c66101 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:01:52.818295 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6abb684d-7b77-44b5-8329-7527d0c66101 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:01:52.830814 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb684d-7b77-44b5-8329-7527d0c66101 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:01:52.832430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb684d-7b77-44b5-8329-7527d0c66101 service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:01:52.842567 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1244/2500] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:01:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:02:02.838472 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=58 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:02:43.376860 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:02:43.378659 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:02:43.456924 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc306147-8d61-4673-aa5d-da1dc9ca7196 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:02:43.462961 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3e3f6-bdc9-4b61-a4af-d096b6e49359 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e13692c1e83413a909d99bb7db2e4b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:02:43.463580 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3e3f6-bdc9-4b61-a4af-d096b6e49359 service nova] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:02:43.463863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3e3f6-bdc9-4b61-a4af-d096b6e49359 service nova] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:02:43.464184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3e3f6-bdc9-4b61-a4af-d096b6e49359 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:02:43.466051 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-91a3e3f6-bdc9-4b61-a4af-d096b6e49359 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:02:43.529288 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3e3f6-bdc9-4b61-a4af-d096b6e49359 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:02:43.530905 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3e3f6-bdc9-4b61-a4af-d096b6e49359 service nova] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:02:43.540014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1257/2501] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:02:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:02:53.535808 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=56 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:03:02.867709 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d6abf-ed56-469b-b0d2-dc8bf8f6d91c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:03:02.868303 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d6abf-ed56-469b-b0d2-dc8bf8f6d91c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:03:02.868660 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d6abf-ed56-469b-b0d2-dc8bf8f6d91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:03:02.889131 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c89d6abf-ed56-469b-b0d2-dc8bf8f6d91c None None] MFA Rules not processed for user `a10fc1a48c9c41eaa1bf3ff2e4109657`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:03:02.890340 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c89d6abf-ed56-469b-b0d2-dc8bf8f6d91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:03:02.898055 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1245/2502] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:03:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:03:03.493014 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3e3f6-bdc9-4b61-a4af-d096b6e49359 service nova] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:03:03.494622 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-91a3e3f6-bdc9-4b61-a4af-d096b6e49359 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:03:03.560895 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3e3f6-bdc9-4b61-a4af-d096b6e49359 service nova] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:03:03.566254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31027ae6-dfb3-4290-9f4e-fb121529ea96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:03:03.566890 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31027ae6-dfb3-4290-9f4e-fb121529ea96 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:03:03.567170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31027ae6-dfb3-4290-9f4e-fb121529ea96 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:03:03.567444 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31027ae6-dfb3-4290-9f4e-fb121529ea96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:03:03.569769 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-31027ae6-dfb3-4290-9f4e-fb121529ea96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:03:03.642087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31027ae6-dfb3-4290-9f4e-fb121529ea96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:03:03.643953 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31027ae6-dfb3-4290-9f4e-fb121529ea96 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:03:03.654090 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1258/2503] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:03:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:03:07.184530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d6abf-ed56-469b-b0d2-dc8bf8f6d91c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:03:07.186600 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c89d6abf-ed56-469b-b0d2-dc8bf8f6d91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:03:07.203512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d6abf-ed56-469b-b0d2-dc8bf8f6d91c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:03:07.210508 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8a08d-4241-43cc-b16d-3b7ba2afe6d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6720518c03413aa8f85e4d6142fe87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:03:07.211479 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8a08d-4241-43cc-b16d-3b7ba2afe6d2 service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:03:07.211959 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8a08d-4241-43cc-b16d-3b7ba2afe6d2 service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:03:07.212079 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8a08d-4241-43cc-b16d-3b7ba2afe6d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:03:07.213802 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f8b8a08d-4241-43cc-b16d-3b7ba2afe6d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:03:07.227165 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8a08d-4241-43cc-b16d-3b7ba2afe6d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:03:07.230291 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8a08d-4241-43cc-b16d-3b7ba2afe6d2 service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:03:07.248414 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1246/2504] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:03:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:03:11.172290 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31027ae6-dfb3-4290-9f4e-fb121529ea96 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:03:11.174507 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-31027ae6-dfb3-4290-9f4e-fb121529ea96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:03:11.189568 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31027ae6-dfb3-4290-9f4e-fb121529ea96 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:03:11.194423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839a83f-4e5f-43ed-9aec-1d7db80a59b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:03:11.195134 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839a83f-4e5f-43ed-9aec-1d7db80a59b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:03:11.195321 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839a83f-4e5f-43ed-9aec-1d7db80a59b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:03:11.195765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839a83f-4e5f-43ed-9aec-1d7db80a59b0 admin admin] PATH_INFO: `/v3/users/193bf7a17f544770a27c07f6b8cc0988` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:03:11.215309 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839a83f-4e5f-43ed-9aec-1d7db80a59b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=193bf7a17f544770a27c07f6b8cc0988)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:03:11.218934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839a83f-4e5f-43ed-9aec-1d7db80a59b0 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:03:11.274370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2839a83f-4e5f-43ed-9aec-1d7db80a59b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '193bf7a17f544770a27c07f6b8cc0988'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.284026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2839a83f-4e5f-43ed-9aec-1d7db80a59b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '193bf7a17f544770a27c07f6b8cc0988'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.284728 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2839a83f-4e5f-43ed-9aec-1d7db80a59b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '193bf7a17f544770a27c07f6b8cc0988'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.285353 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2839a83f-4e5f-43ed-9aec-1d7db80a59b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '193bf7a17f544770a27c07f6b8cc0988'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.294035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2839a83f-4e5f-43ed-9aec-1d7db80a59b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '193bf7a17f544770a27c07f6b8cc0988'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.311751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1259/2505] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:03:11 2022] DELETE /identity/v3/users/193bf7a17f544770a27c07f6b8cc0988 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:03:11.498251 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8a08d-4241-43cc-b16d-3b7ba2afe6d2 service placement] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:03:11.518768 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8a08d-4241-43cc-b16d-3b7ba2afe6d2 service placement] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:03:11.588366 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:03:11.589261 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:03:11.589633 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:03:11.590741 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] PATH_INFO: `/v3/projects/243c0b9b4bd742659061745c8dc4c696` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:03:11.605273 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=243c0b9b4bd742659061745c8dc4c696)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:03:11.607239 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:03:11.666775 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243c0b9b4bd742659061745c8dc4c696', 'request_id': 'req-ffadc306-7ff4-439d-b15a-4327ed8b4425'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.674706 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243c0b9b4bd742659061745c8dc4c696', 'request_id': 'req-ffadc306-7ff4-439d-b15a-4327ed8b4425'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.683826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243c0b9b4bd742659061745c8dc4c696', 'request_id': 'req-ffadc306-7ff4-439d-b15a-4327ed8b4425'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.695443 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] The token cache is being invalidate because project 243c0b9b4bd742659061745c8dc4c696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:03:11.695891 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.696642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.698321 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1247/2506] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:03:11 2022] DELETE /identity/v3/projects/243c0b9b4bd742659061745c8dc4c696 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:03:11.702714 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839a83f-4e5f-43ed-9aec-1d7db80a59b0 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:03:11.704397 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2839a83f-4e5f-43ed-9aec-1d7db80a59b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:03:11.719154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839a83f-4e5f-43ed-9aec-1d7db80a59b0 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:03:11.732800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:03:11.784873 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1085e78-c285-4347-8ade-5cf70e902373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:03:11.785673 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1085e78-c285-4347-8ade-5cf70e902373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:03:11.785954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1085e78-c285-4347-8ade-5cf70e902373 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:03:11.786216 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1085e78-c285-4347-8ade-5cf70e902373 admin admin] PATH_INFO: `/v3/users/d36a68a6a2c240988aaa151bfe43fe1e` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:03:11.800885 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1085e78-c285-4347-8ade-5cf70e902373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d36a68a6a2c240988aaa151bfe43fe1e)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:03:11.802843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1085e78-c285-4347-8ade-5cf70e902373 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:03:11.867045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f1085e78-c285-4347-8ade-5cf70e902373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd36a68a6a2c240988aaa151bfe43fe1e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.874625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f1085e78-c285-4347-8ade-5cf70e902373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd36a68a6a2c240988aaa151bfe43fe1e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.875338 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f1085e78-c285-4347-8ade-5cf70e902373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd36a68a6a2c240988aaa151bfe43fe1e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.875960 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f1085e78-c285-4347-8ade-5cf70e902373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd36a68a6a2c240988aaa151bfe43fe1e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.885532 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f1085e78-c285-4347-8ade-5cf70e902373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd36a68a6a2c240988aaa151bfe43fe1e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:11.903541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1260/2507] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:03:11 2022] DELETE /identity/v3/users/d36a68a6a2c240988aaa151bfe43fe1e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:03:12.082409 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:03:12.084066 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:03:12.098013 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadc306-7ff4-439d-b15a-4327ed8b4425 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:03:12.159341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:03:12.159898 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:03:12.160468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:03:12.160743 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] PATH_INFO: `/v3/projects/52d0343fa71c4c179a2178b88e6def9c` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:03:12.163502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:03:12.172720 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d0343fa71c4c179a2178b88e6def9c)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:03:12.175183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:03:12.231187 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d0343fa71c4c179a2178b88e6def9c', 'request_id': 'req-bb142845-b674-49f0-9a04-012afd28e2a9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:12.239163 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d0343fa71c4c179a2178b88e6def9c', 'request_id': 'req-bb142845-b674-49f0-9a04-012afd28e2a9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:12.249427 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d0343fa71c4c179a2178b88e6def9c', 'request_id': 'req-bb142845-b674-49f0-9a04-012afd28e2a9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:12.261256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] The token cache is being invalidate because project 52d0343fa71c4c179a2178b88e6def9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:03:12.261544 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:12.262224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:03:12.263767 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1248/2508] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:03:12 2022] DELETE /identity/v3/projects/52d0343fa71c4c179a2178b88e6def9c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:03:21.899753 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:03:22.257706 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:04:31.689305 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1085e78-c285-4347-8ade-5cf70e902373 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:31.691801 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-f1085e78-c285-4347-8ade-5cf70e902373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:04:31.764980 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1085e78-c285-4347-8ade-5cf70e902373 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:31.825765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4e73a-c6b6-4537-8e41-a1f898be1ce2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:31.826280 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4e73a-c6b6-4537-8e41-a1f898be1ce2 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:31.826560 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4e73a-c6b6-4537-8e41-a1f898be1ce2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:31.826831 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4e73a-c6b6-4537-8e41-a1f898be1ce2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:31.828659 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-39a4e73a-c6b6-4537-8e41-a1f898be1ce2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:04:31.908201 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4e73a-c6b6-4537-8e41-a1f898be1ce2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:31.909654 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4e73a-c6b6-4537-8e41-a1f898be1ce2 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:31.921521 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1261/2509] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:04:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:04:38.881800 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:38.883310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:04:38.898729 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb142845-b674-49f0-9a04-012afd28e2a9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:38.903430 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b24221-3240-4d8f-a751-1186fd83405b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:38.904194 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b24221-3240-4d8f-a751-1186fd83405b service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:38.904615 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b24221-3240-4d8f-a751-1186fd83405b service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:38.904885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b24221-3240-4d8f-a751-1186fd83405b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:38.906822 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a3b24221-3240-4d8f-a751-1186fd83405b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:04:38.921280 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b24221-3240-4d8f-a751-1186fd83405b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:38.922803 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b24221-3240-4d8f-a751-1186fd83405b service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:38.936606 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1249/2510] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:04:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:04:41.916162 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=74 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:04:48.430573 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4e73a-c6b6-4537-8e41-a1f898be1ce2 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:48.432194 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-39a4e73a-c6b6-4537-8e41-a1f898be1ce2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:04:48.491331 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4e73a-c6b6-4537-8e41-a1f898be1ce2 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:48.495772 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01227803-6bf8-446c-9e28-bf011c652dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:48.496417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01227803-6bf8-446c-9e28-bf011c652dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:48.496695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01227803-6bf8-446c-9e28-bf011c652dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:48.496965 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01227803-6bf8-446c-9e28-bf011c652dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:48.499079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-01227803-6bf8-446c-9e28-bf011c652dd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:04:48.571948 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01227803-6bf8-446c-9e28-bf011c652dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:48.573419 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01227803-6bf8-446c-9e28-bf011c652dd1 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:48.587510 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1262/2511] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:04:48.929299 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=6 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:04:52.038298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b24221-3240-4d8f-a751-1186fd83405b service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:52.038298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a3b24221-3240-4d8f-a751-1186fd83405b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:04:52.053955 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b24221-3240-4d8f-a751-1186fd83405b service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:52.060257 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e76f81-a221-4110-9aa2-04121e6c36a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:52.061226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e76f81-a221-4110-9aa2-04121e6c36a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:52.061507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e76f81-a221-4110-9aa2-04121e6c36a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:52.061774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e76f81-a221-4110-9aa2-04121e6c36a2 admin admin] PATH_INFO: `/v3/users/a10fc1a48c9c41eaa1bf3ff2e4109657` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:52.062887 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e76f81-a221-4110-9aa2-04121e6c36a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a10fc1a48c9c41eaa1bf3ff2e4109657)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:52.064849 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e76f81-a221-4110-9aa2-04121e6c36a2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:52.116030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-90e76f81-a221-4110-9aa2-04121e6c36a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a10fc1a48c9c41eaa1bf3ff2e4109657'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:04:52.116761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-90e76f81-a221-4110-9aa2-04121e6c36a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a10fc1a48c9c41eaa1bf3ff2e4109657'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:04:52.117410 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-90e76f81-a221-4110-9aa2-04121e6c36a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a10fc1a48c9c41eaa1bf3ff2e4109657'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:04:52.129950 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-90e76f81-a221-4110-9aa2-04121e6c36a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a10fc1a48c9c41eaa1bf3ff2e4109657'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:04:52.139505 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-90e76f81-a221-4110-9aa2-04121e6c36a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a10fc1a48c9c41eaa1bf3ff2e4109657'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:04:52.152690 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1250/2512] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:04:52 2022] DELETE /identity/v3/users/a10fc1a48c9c41eaa1bf3ff2e4109657 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:04:52.334762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01227803-6bf8-446c-9e28-bf011c652dd1 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:52.345893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01227803-6bf8-446c-9e28-bf011c652dd1 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:52.416542 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:52.417234 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:52.417511 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:52.417782 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0 admin admin] PATH_INFO: `/v3/projects/1976b27e84a4484ba9d46a4244b08699` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:52.419392 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1976b27e84a4484ba9d46a4244b08699)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:52.421249 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:52.477099 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1976b27e84a4484ba9d46a4244b08699', 'request_id': 'req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:04:52.479692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:04:52.487432 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1976b27e84a4484ba9d46a4244b08699', 'request_id': 'req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:04:52.495887 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1976b27e84a4484ba9d46a4244b08699', 'request_id': 'req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:04:52.507380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0 admin admin] The token cache is being invalidate because project 1976b27e84a4484ba9d46a4244b08699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:04:52.507668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:04:52.508520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-2c41054b-3a60-4ea5-a9b6-40d59fe42ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:04:52.510316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1263/2513] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:04:52 2022] DELETE /identity/v3/projects/1976b27e84a4484ba9d46a4244b08699 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:04:52.515675 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06faa67-e1b8-4cf9-beee-a536dfa2533b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:52.516134 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06faa67-e1b8-4cf9-beee-a536dfa2533b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:52.516497 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06faa67-e1b8-4cf9-beee-a536dfa2533b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:52.537671 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b06faa67-e1b8-4cf9-beee-a536dfa2533b None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:04:52.538903 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b06faa67-e1b8-4cf9-beee-a536dfa2533b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:04:52.600270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1251/2514] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:04:52.609579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b21c9f-8c0f-4897-b1f3-fc798395fb97 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:52.609828 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b21c9f-8c0f-4897-b1f3-fc798395fb97 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:52.610107 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b21c9f-8c0f-4897-b1f3-fc798395fb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:52.633448 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-46b21c9f-8c0f-4897-b1f3-fc798395fb97 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:04:52.634655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-46b21c9f-8c0f-4897-b1f3-fc798395fb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:04:52.643652 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1264/2515] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:04:52.650125 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06faa67-e1b8-4cf9-beee-a536dfa2533b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:52.662940 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06faa67-e1b8-4cf9-beee-a536dfa2533b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:52.668490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71792c5f-2fe6-406b-9203-e9fa931a4e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:52.669097 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71792c5f-2fe6-406b-9203-e9fa931a4e94 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:52.669370 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71792c5f-2fe6-406b-9203-e9fa931a4e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:52.669634 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71792c5f-2fe6-406b-9203-e9fa931a4e94 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:52.670597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71792c5f-2fe6-406b-9203-e9fa931a4e94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:04:52.670997 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71792c5f-2fe6-406b-9203-e9fa931a4e94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:52.672597 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71792c5f-2fe6-406b-9203-e9fa931a4e94 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:52.684970 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1252/2516] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 14:04:52.690172 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b21c9f-8c0f-4897-b1f3-fc798395fb97 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:52.700473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b21c9f-8c0f-4897-b1f3-fc798395fb97 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:52.705987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbfb09-63c0-49da-8930-21d1b18697c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:52.706591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbfb09-63c0-49da-8930-21d1b18697c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:52.706870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbfb09-63c0-49da-8930-21d1b18697c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:52.707148 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbfb09-63c0-49da-8930-21d1b18697c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:52.707764 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbfb09-63c0-49da-8930-21d1b18697c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:52.709701 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbfb09-63c0-49da-8930-21d1b18697c6 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:52.739508 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1265/2517] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:04:52 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:04:52.743844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71792c5f-2fe6-406b-9203-e9fa931a4e94 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:52.754438 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71792c5f-2fe6-406b-9203-e9fa931a4e94 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:52.834032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd053f-6d0b-456b-aacb-c86a4606ed04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:52.834543 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd053f-6d0b-456b-aacb-c86a4606ed04 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:52.834823 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd053f-6d0b-456b-aacb-c86a4606ed04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:52.835091 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd053f-6d0b-456b-aacb-c86a4606ed04 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:52.835728 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd053f-6d0b-456b-aacb-c86a4606ed04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:52.837667 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd053f-6d0b-456b-aacb-c86a4606ed04 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:52.846143 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:04:52.864723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1253/2518] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:04:52 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:04:52.869001 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbfb09-63c0-49da-8930-21d1b18697c6 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:52.880526 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbfb09-63c0-49da-8930-21d1b18697c6 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:52.886598 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a532e0-f6dc-4478-afa7-fb96bc792043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:52.887217 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a532e0-f6dc-4478-afa7-fb96bc792043 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:52.887487 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a532e0-f6dc-4478-afa7-fb96bc792043 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:52.887731 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a532e0-f6dc-4478-afa7-fb96bc792043 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:52.888417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a532e0-f6dc-4478-afa7-fb96bc792043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:52.890501 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a532e0-f6dc-4478-afa7-fb96bc792043 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:52.901270 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1266/2519] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:04:52.905738 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd053f-6d0b-456b-aacb-c86a4606ed04 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:52.916646 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd053f-6d0b-456b-aacb-c86a4606ed04 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:52.922265 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e12c0-ac98-465c-aa71-184123a5e2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:52.922875 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e12c0-ac98-465c-aa71-184123a5e2ac admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:52.923151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e12c0-ac98-465c-aa71-184123a5e2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:52.923423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e12c0-ac98-465c-aa71-184123a5e2ac admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:52.924107 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e12c0-ac98-465c-aa71-184123a5e2ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:52.925751 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e12c0-ac98-465c-aa71-184123a5e2ac admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:52.936787 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1254/2520] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:04:52.941109 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a532e0-f6dc-4478-afa7-fb96bc792043 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:52.951218 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a532e0-f6dc-4478-afa7-fb96bc792043 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:52.956591 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbe66e-e984-4043-bcc5-44940aef1637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:52.957281 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbe66e-e984-4043-bcc5-44940aef1637 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:52.957530 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbe66e-e984-4043-bcc5-44940aef1637 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:52.957815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbe66e-e984-4043-bcc5-44940aef1637 admin admin] PATH_INFO: `/v3/projects/ca5fb1daf92a4583bf8fef26e422b7e9/users/bb24160e8a9d4da984c95b6e2be6cece/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:52.974884 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bbe66e-e984-4043-bcc5-44940aef1637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5fb1daf92a4583bf8fef26e422b7e9, user_id=bb24160e8a9d4da984c95b6e2be6cece, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:52.977569 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bbe66e-e984-4043-bcc5-44940aef1637 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:52.993835 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1267/2521] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:04:52 2022] PUT /identity/v3/projects/ca5fb1daf92a4583bf8fef26e422b7e9/users/bb24160e8a9d4da984c95b6e2be6cece/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:04:53.005493 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6e8e3-5054-4de5-9fc7-6d4a374813fa None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:53.005776 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6e8e3-5054-4de5-9fc7-6d4a374813fa None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:53.006045 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6e8e3-5054-4de5-9fc7-6d4a374813fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:53.036198 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8ec6e8e3-5054-4de5-9fc7-6d4a374813fa None None] MFA Rules not processed for user `bb24160e8a9d4da984c95b6e2be6cece`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:04:53.037406 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8ec6e8e3-5054-4de5-9fc7-6d4a374813fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:04:53.104122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1255/2522] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:04:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:04:53.108921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbe66e-e984-4043-bcc5-44940aef1637 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:53.179974 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbe66e-e984-4043-bcc5-44940aef1637 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:53.186401 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaa92c-3836-4e34-b0a4-d3571f250fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:53.187482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaa92c-3836-4e34-b0a4-d3571f250fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:53.187769 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaa92c-3836-4e34-b0a4-d3571f250fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:53.188099 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaa92c-3836-4e34-b0a4-d3571f250fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:53.188740 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbaa92c-3836-4e34-b0a4-d3571f250fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:53.190640 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbaa92c-3836-4e34-b0a4-d3571f250fb0 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:53.220873 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1268/2523] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:04:53 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:04:53.225044 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6e8e3-5054-4de5-9fc7-6d4a374813fa None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:53.237018 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6e8e3-5054-4de5-9fc7-6d4a374813fa None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:53.299964 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9868ed-12b0-4e49-8c52-3d5dbb31d614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:53.300578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9868ed-12b0-4e49-8c52-3d5dbb31d614 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:53.300844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9868ed-12b0-4e49-8c52-3d5dbb31d614 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:53.301111 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9868ed-12b0-4e49-8c52-3d5dbb31d614 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:53.301745 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9868ed-12b0-4e49-8c52-3d5dbb31d614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:53.303676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9868ed-12b0-4e49-8c52-3d5dbb31d614 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:53.328010 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1256/2524] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:04:53 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 14:04:53.332079 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaa92c-3836-4e34-b0a4-d3571f250fb0 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:53.342761 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaa92c-3836-4e34-b0a4-d3571f250fb0 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:53.349091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcc1ea-7f29-48f1-9ade-064678acfdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:53.349710 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcc1ea-7f29-48f1-9ade-064678acfdb4 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:53.350044 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcc1ea-7f29-48f1-9ade-064678acfdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:53.350318 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcc1ea-7f29-48f1-9ade-064678acfdb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:53.350973 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcc1ea-7f29-48f1-9ade-064678acfdb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:53.352841 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcc1ea-7f29-48f1-9ade-064678acfdb4 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:53.365071 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1269/2525] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:04:53.369865 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9868ed-12b0-4e49-8c52-3d5dbb31d614 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:53.381067 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9868ed-12b0-4e49-8c52-3d5dbb31d614 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:53.386434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3eada-9bb4-46fb-9ef3-e8412728eb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:53.387095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3eada-9bb4-46fb-9ef3-e8412728eb00 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:53.387366 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3eada-9bb4-46fb-9ef3-e8412728eb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:53.387664 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3eada-9bb4-46fb-9ef3-e8412728eb00 admin admin] PATH_INFO: `/v3/projects/dc7ade5682d844fc87732acef4e218fa/users/207bb32254a64f138512580dec597c7f/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:53.405763 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3eada-9bb4-46fb-9ef3-e8412728eb00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7ade5682d844fc87732acef4e218fa, user_id=207bb32254a64f138512580dec597c7f, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:53.408605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3eada-9bb4-46fb-9ef3-e8412728eb00 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:53.423977 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1257/2526] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:04:53 2022] PUT /identity/v3/projects/dc7ade5682d844fc87732acef4e218fa/users/207bb32254a64f138512580dec597c7f/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:04:53.428332 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcc1ea-7f29-48f1-9ade-064678acfdb4 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:53.439409 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcc1ea-7f29-48f1-9ade-064678acfdb4 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:53.488038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:04:53.499333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b8a20-888a-4801-817a-d44bb9ff104d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:53.500045 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b8a20-888a-4801-817a-d44bb9ff104d admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:53.500559 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b8a20-888a-4801-817a-d44bb9ff104d admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:53.500895 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b8a20-888a-4801-817a-d44bb9ff104d admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:53.501694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b8a20-888a-4801-817a-d44bb9ff104d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:53.503538 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b8a20-888a-4801-817a-d44bb9ff104d admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:53.514435 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1270/2527] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:04:53.518878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3eada-9bb4-46fb-9ef3-e8412728eb00 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:53.529068 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3eada-9bb4-46fb-9ef3-e8412728eb00 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:53.533595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0f2f9-6f2b-48a6-95c5-18fee6239c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:53.534239 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0f2f9-6f2b-48a6-95c5-18fee6239c83 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:53.534509 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0f2f9-6f2b-48a6-95c5-18fee6239c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:53.534769 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0f2f9-6f2b-48a6-95c5-18fee6239c83 admin admin] PATH_INFO: `/v3/projects/dc7ade5682d844fc87732acef4e218fa/users/207bb32254a64f138512580dec597c7f/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:53.536676 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0f2f9-6f2b-48a6-95c5-18fee6239c83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7ade5682d844fc87732acef4e218fa, user_id=207bb32254a64f138512580dec597c7f, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:53.539284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0f2f9-6f2b-48a6-95c5-18fee6239c83 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:53.549420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1258/2528] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:04:53 2022] PUT /identity/v3/projects/dc7ade5682d844fc87732acef4e218fa/users/207bb32254a64f138512580dec597c7f/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:04:53.562954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500fd80-225c-4bae-8fcd-479f9193c7c7 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:53.563293 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500fd80-225c-4bae-8fcd-479f9193c7c7 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:53.563624 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500fd80-225c-4bae-8fcd-479f9193c7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:53.596699 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-2500fd80-225c-4bae-8fcd-479f9193c7c7 None None] MFA Rules not processed for user `207bb32254a64f138512580dec597c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:04:53.597941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2500fd80-225c-4bae-8fcd-479f9193c7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:04:53.667968 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1271/2529] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:04:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:04:53.679608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0f2f9-6f2b-48a6-95c5-18fee6239c83 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:53.681587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c1d0f2f9-6f2b-48a6-95c5-18fee6239c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:04:53.706218 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:04:53.743105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0f2f9-6f2b-48a6-95c5-18fee6239c83 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:53.746801 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398edea1-f7a4-4bf8-8004-8cebf427fb2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:53.747402 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398edea1-f7a4-4bf8-8004-8cebf427fb2f service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:53.747671 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398edea1-f7a4-4bf8-8004-8cebf427fb2f service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:53.747971 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398edea1-f7a4-4bf8-8004-8cebf427fb2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:53.757713 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398edea1-f7a4-4bf8-8004-8cebf427fb2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:53.759140 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398edea1-f7a4-4bf8-8004-8cebf427fb2f service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:53.768730 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1259/2530] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:04:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:04:53.863135 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500fd80-225c-4bae-8fcd-479f9193c7c7 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:04:53.873346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500fd80-225c-4bae-8fcd-479f9193c7c7 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:04:53.877751 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb31b54-4c26-4290-93ff-b3addce7df30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:04:53.878365 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb31b54-4c26-4290-93ff-b3addce7df30 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:04:53.878635 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb31b54-4c26-4290-93ff-b3addce7df30 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:04:53.878897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb31b54-4c26-4290-93ff-b3addce7df30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:04:53.939238 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb31b54-4c26-4290-93ff-b3addce7df30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:04:53.940614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb31b54-4c26-4290-93ff-b3addce7df30 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:04:53.949518 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1272/2531] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:04:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:05:03.764364 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=17 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:03.945541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=68 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:04.176765 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398edea1-f7a4-4bf8-8004-8cebf427fb2f service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:04.251095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398edea1-f7a4-4bf8-8004-8cebf427fb2f service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:04.255851 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a57bd-6d7c-41d2-8544-2218153a0df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:04.256520 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a57bd-6d7c-41d2-8544-2218153a0df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:04.256804 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a57bd-6d7c-41d2-8544-2218153a0df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:04.257076 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a57bd-6d7c-41d2-8544-2218153a0df5 admin admin] PATH_INFO: `/v3/users/bb24160e8a9d4da984c95b6e2be6cece` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:04.258212 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a57bd-6d7c-41d2-8544-2218153a0df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb24160e8a9d4da984c95b6e2be6cece)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:04.260199 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a57bd-6d7c-41d2-8544-2218153a0df5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:04.311666 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-472a57bd-6d7c-41d2-8544-2218153a0df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb24160e8a9d4da984c95b6e2be6cece'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:04.312270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-472a57bd-6d7c-41d2-8544-2218153a0df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb24160e8a9d4da984c95b6e2be6cece'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:04.312927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-472a57bd-6d7c-41d2-8544-2218153a0df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb24160e8a9d4da984c95b6e2be6cece'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:04.327794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-472a57bd-6d7c-41d2-8544-2218153a0df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb24160e8a9d4da984c95b6e2be6cece'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:04.337105 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-472a57bd-6d7c-41d2-8544-2218153a0df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb24160e8a9d4da984c95b6e2be6cece'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:04.350649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1260/2532] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:05:04 2022] DELETE /identity/v3/users/bb24160e8a9d4da984c95b6e2be6cece => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:04.356135 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a216e-b7c8-4654-947c-b03c4435a370 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:04.356515 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a216e-b7c8-4654-947c-b03c4435a370 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:04.356887 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a216e-b7c8-4654-947c-b03c4435a370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:04.377815 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-457a216e-b7c8-4654-947c-b03c4435a370 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:04.379107 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-457a216e-b7c8-4654-947c-b03c4435a370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:04.441499 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1273/2533] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:04.451642 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a57bd-6d7c-41d2-8544-2218153a0df5 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:04.454034 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-472a57bd-6d7c-41d2-8544-2218153a0df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:04.522253 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a57bd-6d7c-41d2-8544-2218153a0df5 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:04.527171 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e8615-261c-43a1-9001-4a60e95031b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:04.528178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e8615-261c-43a1-9001-4a60e95031b4 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:04.528545 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e8615-261c-43a1-9001-4a60e95031b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:04.528799 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e8615-261c-43a1-9001-4a60e95031b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:04.543002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e8615-261c-43a1-9001-4a60e95031b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:04.544495 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e8615-261c-43a1-9001-4a60e95031b4 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:04.555178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1261/2534] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:05:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 15 14:05:04.844725 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a216e-b7c8-4654-947c-b03c4435a370 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:04.846552 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-457a216e-b7c8-4654-947c-b03c4435a370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:04.860899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a216e-b7c8-4654-947c-b03c4435a370 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:04.865917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:04.867223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:04.867223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:04.867223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] PATH_INFO: `/v3/projects/ca5fb1daf92a4583bf8fef26e422b7e9` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:04.868295 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5fb1daf92a4583bf8fef26e422b7e9)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:04.874823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:04.919277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5fb1daf92a4583bf8fef26e422b7e9', 'request_id': 'req-ddb069a7-6c22-491e-8a04-e8545bf0e075'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:04.927154 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5fb1daf92a4583bf8fef26e422b7e9', 'request_id': 'req-ddb069a7-6c22-491e-8a04-e8545bf0e075'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:04.933757 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5fb1daf92a4583bf8fef26e422b7e9', 'request_id': 'req-ddb069a7-6c22-491e-8a04-e8545bf0e075'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:04.944534 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] The token cache is being invalidate because project ca5fb1daf92a4583bf8fef26e422b7e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:05:04.944823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:04.945498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:04.947004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1274/2535] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:05:04 2022] DELETE /identity/v3/projects/ca5fb1daf92a4583bf8fef26e422b7e9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:04.952617 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e8615-261c-43a1-9001-4a60e95031b4 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:04.963178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e8615-261c-43a1-9001-4a60e95031b4 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:04.983120 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:05.022431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402c805-4830-4631-9241-9ad8f3dd94fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:05.023133 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402c805-4830-4631-9241-9ad8f3dd94fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:05.023393 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402c805-4830-4631-9241-9ad8f3dd94fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:05.023656 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402c805-4830-4631-9241-9ad8f3dd94fc admin admin] PATH_INFO: `/v3/users/207bb32254a64f138512580dec597c7f` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:05.024824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402c805-4830-4631-9241-9ad8f3dd94fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=207bb32254a64f138512580dec597c7f)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:05.026766 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402c805-4830-4631-9241-9ad8f3dd94fc admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:05.075164 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e402c805-4830-4631-9241-9ad8f3dd94fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '207bb32254a64f138512580dec597c7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:05.075827 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e402c805-4830-4631-9241-9ad8f3dd94fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '207bb32254a64f138512580dec597c7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:05.076491 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e402c805-4830-4631-9241-9ad8f3dd94fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '207bb32254a64f138512580dec597c7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:05.090857 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e402c805-4830-4631-9241-9ad8f3dd94fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '207bb32254a64f138512580dec597c7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:05.100552 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e402c805-4830-4631-9241-9ad8f3dd94fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '207bb32254a64f138512580dec597c7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:05.118421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1262/2536] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:05:04 2022] DELETE /identity/v3/users/207bb32254a64f138512580dec597c7f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:05.403402 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:05.404979 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:05.419477 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb069a7-6c22-491e-8a04-e8545bf0e075 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:05.489021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce486549-bda3-4125-aa77-0e0ee769eab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:05.489668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce486549-bda3-4125-aa77-0e0ee769eab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:05.489923 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce486549-bda3-4125-aa77-0e0ee769eab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:05.490225 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce486549-bda3-4125-aa77-0e0ee769eab6 admin admin] PATH_INFO: `/v3/projects/dc7ade5682d844fc87732acef4e218fa` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:05.491387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce486549-bda3-4125-aa77-0e0ee769eab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc7ade5682d844fc87732acef4e218fa)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:05.493335 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce486549-bda3-4125-aa77-0e0ee769eab6 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:05.525629 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:05.543358 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ce486549-bda3-4125-aa77-0e0ee769eab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc7ade5682d844fc87732acef4e218fa', 'request_id': 'req-ce486549-bda3-4125-aa77-0e0ee769eab6'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:05.552492 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ce486549-bda3-4125-aa77-0e0ee769eab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc7ade5682d844fc87732acef4e218fa', 'request_id': 'req-ce486549-bda3-4125-aa77-0e0ee769eab6'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:05.559623 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ce486549-bda3-4125-aa77-0e0ee769eab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc7ade5682d844fc87732acef4e218fa', 'request_id': 'req-ce486549-bda3-4125-aa77-0e0ee769eab6'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:05.570735 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ce486549-bda3-4125-aa77-0e0ee769eab6 admin admin] The token cache is being invalidate because project dc7ade5682d844fc87732acef4e218fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87915) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:05:05.571052 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ce486549-bda3-4125-aa77-0e0ee769eab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:05.571727 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-ce486549-bda3-4125-aa77-0e0ee769eab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:05.573274 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1275/2537] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:05:05 2022] DELETE /identity/v3/projects/dc7ade5682d844fc87732acef4e218fa => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:05.579053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137c34a-fcce-4475-b5a8-4a72069826b7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:05.579424 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137c34a-fcce-4475-b5a8-4a72069826b7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:05.579833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137c34a-fcce-4475-b5a8-4a72069826b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:05.606587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c137c34a-fcce-4475-b5a8-4a72069826b7 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:05.607820 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c137c34a-fcce-4475-b5a8-4a72069826b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:05.682239 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1263/2538] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:05.692051 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e8eb1-09bd-442f-a44b-c4fff14bb43e None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:05.692306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e8eb1-09bd-442f-a44b-c4fff14bb43e None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:05.692614 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e8eb1-09bd-442f-a44b-c4fff14bb43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:05.716750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-238e8eb1-09bd-442f-a44b-c4fff14bb43e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:05.718004 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-238e8eb1-09bd-442f-a44b-c4fff14bb43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:05.727414 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1276/2539] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:05.732773 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137c34a-fcce-4475-b5a8-4a72069826b7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:05.745510 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137c34a-fcce-4475-b5a8-4a72069826b7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:05.754318 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba78af-f907-48f8-a531-fb4d72d98b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:05.755256 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba78af-f907-48f8-a531-fb4d72d98b52 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:05.755658 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba78af-f907-48f8-a531-fb4d72d98b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:05.756181 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba78af-f907-48f8-a531-fb4d72d98b52 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:05.757255 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba78af-f907-48f8-a531-fb4d72d98b52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:05:05.757669 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba78af-f907-48f8-a531-fb4d72d98b52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:05.760132 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba78af-f907-48f8-a531-fb4d72d98b52 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:05.775606 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1264/2540] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 14:05:05.780431 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e8eb1-09bd-442f-a44b-c4fff14bb43e None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:05.789987 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e8eb1-09bd-442f-a44b-c4fff14bb43e None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:05.795312 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c1001-f6e7-4a89-ab4f-1e837332cb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:05.795944 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c1001-f6e7-4a89-ab4f-1e837332cb5b admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:05.796216 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c1001-f6e7-4a89-ab4f-1e837332cb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:05.796504 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c1001-f6e7-4a89-ab4f-1e837332cb5b admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:05.797139 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c1001-f6e7-4a89-ab4f-1e837332cb5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:05.799040 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c1001-f6e7-4a89-ab4f-1e837332cb5b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:05.830101 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1277/2541] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:05:05 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:05:05.835120 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba78af-f907-48f8-a531-fb4d72d98b52 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:05.845423 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba78af-f907-48f8-a531-fb4d72d98b52 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:05.875192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:05.906215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ac659-0625-421f-80af-dde27f8ed39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:05.906825 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ac659-0625-421f-80af-dde27f8ed39c admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:05.907095 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ac659-0625-421f-80af-dde27f8ed39c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:05.907357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ac659-0625-421f-80af-dde27f8ed39c admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:05.908002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ac659-0625-421f-80af-dde27f8ed39c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:05.909972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ac659-0625-421f-80af-dde27f8ed39c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:05.933037 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1265/2542] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:05:05 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:05:05.937463 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c1001-f6e7-4a89-ab4f-1e837332cb5b admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:05.948636 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c1001-f6e7-4a89-ab4f-1e837332cb5b admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:05.954035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c0371-3afa-4644-848b-bde090c1f8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:05.954646 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c0371-3afa-4644-848b-bde090c1f8cc admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:05.954924 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c0371-3afa-4644-848b-bde090c1f8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:05.955199 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c0371-3afa-4644-848b-bde090c1f8cc admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:05.955843 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c0371-3afa-4644-848b-bde090c1f8cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:05.957553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c0371-3afa-4644-848b-bde090c1f8cc admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:05.969441 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1278/2543] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:05:05.973927 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ac659-0625-421f-80af-dde27f8ed39c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:05.984719 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ac659-0625-421f-80af-dde27f8ed39c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:05.990884 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58859ea-b107-442f-98ce-aa58fd1d408a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:05.991494 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58859ea-b107-442f-98ce-aa58fd1d408a admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:05.991772 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58859ea-b107-442f-98ce-aa58fd1d408a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:05.992072 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58859ea-b107-442f-98ce-aa58fd1d408a admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:05.992720 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58859ea-b107-442f-98ce-aa58fd1d408a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:05.994341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58859ea-b107-442f-98ce-aa58fd1d408a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:06.006949 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1266/2544] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 14:05:06.013277 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c0371-3afa-4644-848b-bde090c1f8cc admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:06.026316 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c0371-3afa-4644-848b-bde090c1f8cc admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:06.032491 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c821b3-0385-44fe-88c3-90438d543350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:06.033153 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c821b3-0385-44fe-88c3-90438d543350 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:06.033430 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c821b3-0385-44fe-88c3-90438d543350 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:06.033704 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c821b3-0385-44fe-88c3-90438d543350 admin admin] PATH_INFO: `/v3/projects/7fc47386e20a4b518ca8abd60160d7cc/users/1e9e058ab3e24488965ac3f4639a7072/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:06.052813 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c821b3-0385-44fe-88c3-90438d543350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc47386e20a4b518ca8abd60160d7cc, user_id=1e9e058ab3e24488965ac3f4639a7072, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:06.055490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c821b3-0385-44fe-88c3-90438d543350 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:06.071565 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1279/2545] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:05:06 2022] PUT /identity/v3/projects/7fc47386e20a4b518ca8abd60160d7cc/users/1e9e058ab3e24488965ac3f4639a7072/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:06.082416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715ca9e-db72-43f6-8193-801abf648e25 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:06.082694 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715ca9e-db72-43f6-8193-801abf648e25 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:06.082964 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715ca9e-db72-43f6-8193-801abf648e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:06.112581 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d715ca9e-db72-43f6-8193-801abf648e25 None None] MFA Rules not processed for user `1e9e058ab3e24488965ac3f4639a7072`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:06.113826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d715ca9e-db72-43f6-8193-801abf648e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:06.178635 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1267/2546] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:06.183839 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c821b3-0385-44fe-88c3-90438d543350 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:06.250256 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c821b3-0385-44fe-88c3-90438d543350 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:06.255374 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105477c9-6858-4b93-945a-27d57237a75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:06.256022 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105477c9-6858-4b93-945a-27d57237a75c admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:06.256306 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105477c9-6858-4b93-945a-27d57237a75c admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:06.256581 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105477c9-6858-4b93-945a-27d57237a75c admin admin] PATH_INFO: `/v3/projects` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:06.257389 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105477c9-6858-4b93-945a-27d57237a75c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:06.259310 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105477c9-6858-4b93-945a-27d57237a75c admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:06.288458 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1280/2547] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:05:06 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:05:06.294611 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715ca9e-db72-43f6-8193-801abf648e25 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:06.312231 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715ca9e-db72-43f6-8193-801abf648e25 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:06.374034 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb0489-8ef7-4e31-bc73-2fa57ed4260e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:06.374645 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0489-8ef7-4e31-bc73-2fa57ed4260e admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:06.374921 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0489-8ef7-4e31-bc73-2fa57ed4260e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:06.375190 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0489-8ef7-4e31-bc73-2fa57ed4260e admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:06.375807 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb0489-8ef7-4e31-bc73-2fa57ed4260e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:06.377761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb0489-8ef7-4e31-bc73-2fa57ed4260e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:06.402421 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1268/2548] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:05:06 2022] POST /identity/v3/users => generated 296 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:05:06.406586 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105477c9-6858-4b93-945a-27d57237a75c admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:06.418292 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105477c9-6858-4b93-945a-27d57237a75c admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:06.424092 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a6d68-5903-4131-a7aa-b71dc235ab1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:06.424715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a6d68-5903-4131-a7aa-b71dc235ab1a admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:06.424984 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a6d68-5903-4131-a7aa-b71dc235ab1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:06.425254 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a6d68-5903-4131-a7aa-b71dc235ab1a admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:06.426226 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a6d68-5903-4131-a7aa-b71dc235ab1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:06.427718 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a6d68-5903-4131-a7aa-b71dc235ab1a admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:06.439485 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1281/2549] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:05:06.444106 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb0489-8ef7-4e31-bc73-2fa57ed4260e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:06.455382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb0489-8ef7-4e31-bc73-2fa57ed4260e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:06.460950 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaee476-2a15-4dba-a348-43de104a0241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:06.461602 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaee476-2a15-4dba-a348-43de104a0241 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:06.461879 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaee476-2a15-4dba-a348-43de104a0241 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:06.462144 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaee476-2a15-4dba-a348-43de104a0241 admin admin] PATH_INFO: `/v3/projects/f31fbe2b067445f1b57538a23eddbe98/users/0d66a42dde6d45b382c148a156383b67/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:06.485060 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaee476-2a15-4dba-a348-43de104a0241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f31fbe2b067445f1b57538a23eddbe98, user_id=0d66a42dde6d45b382c148a156383b67, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:06.489216 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaee476-2a15-4dba-a348-43de104a0241 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:06.508085 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1269/2550] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:05:06 2022] PUT /identity/v3/projects/f31fbe2b067445f1b57538a23eddbe98/users/0d66a42dde6d45b382c148a156383b67/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:06.512279 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a6d68-5903-4131-a7aa-b71dc235ab1a admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:06.524925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a6d68-5903-4131-a7aa-b71dc235ab1a admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:06.547042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:06.588647 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc11ee-688a-4f68-b73c-9bafe2058b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:06.589266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc11ee-688a-4f68-b73c-9bafe2058b2b admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:06.589545 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc11ee-688a-4f68-b73c-9bafe2058b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:06.589818 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc11ee-688a-4f68-b73c-9bafe2058b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:06.590465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc11ee-688a-4f68-b73c-9bafe2058b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:06.592091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc11ee-688a-4f68-b73c-9bafe2058b2b admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:06.602686 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1282/2551] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:05:06.607515 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaee476-2a15-4dba-a348-43de104a0241 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:06.613055 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:06.619831 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaee476-2a15-4dba-a348-43de104a0241 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:06.625264 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23b014-9b30-48dc-ae98-ae015f112d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:06.625951 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23b014-9b30-48dc-ae98-ae015f112d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:06.626228 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23b014-9b30-48dc-ae98-ae015f112d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:06.626501 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23b014-9b30-48dc-ae98-ae015f112d94 admin admin] PATH_INFO: `/v3/projects/f31fbe2b067445f1b57538a23eddbe98/users/0d66a42dde6d45b382c148a156383b67/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:06.628477 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23b014-9b30-48dc-ae98-ae015f112d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f31fbe2b067445f1b57538a23eddbe98, user_id=0d66a42dde6d45b382c148a156383b67, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:06.631119 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23b014-9b30-48dc-ae98-ae015f112d94 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:06.642192 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1270/2552] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:05:06 2022] PUT /identity/v3/projects/f31fbe2b067445f1b57538a23eddbe98/users/0d66a42dde6d45b382c148a156383b67/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:06.653661 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366daaa-0c5d-418b-9e74-5c4118e1088f None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:06.653999 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366daaa-0c5d-418b-9e74-5c4118e1088f None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:06.654394 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366daaa-0c5d-418b-9e74-5c4118e1088f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:06.688846 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-c366daaa-0c5d-418b-9e74-5c4118e1088f None None] MFA Rules not processed for user `0d66a42dde6d45b382c148a156383b67`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:06.690145 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-c366daaa-0c5d-418b-9e74-5c4118e1088f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:06.760426 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1283/2553] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:06.773846 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23b014-9b30-48dc-ae98-ae015f112d94 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:06.775663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ac23b014-9b30-48dc-ae98-ae015f112d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:06.844529 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23b014-9b30-48dc-ae98-ae015f112d94 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:06.850459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6d39b-c822-4ea9-88ed-cc3e2281890b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:06.851083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6d39b-c822-4ea9-88ed-cc3e2281890b service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:06.851358 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6d39b-c822-4ea9-88ed-cc3e2281890b service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:06.851629 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6d39b-c822-4ea9-88ed-cc3e2281890b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:06.866394 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6d39b-c822-4ea9-88ed-cc3e2281890b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:06.868475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6d39b-c822-4ea9-88ed-cc3e2281890b service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:06.879752 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1271/2554] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:05:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:05:07.220978 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366daaa-0c5d-418b-9e74-5c4118e1088f None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:07.231253 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366daaa-0c5d-418b-9e74-5c4118e1088f None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:07.237871 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4d939-c31d-46cc-8ff7-3b22370e9ca0 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d66a42dde6d45b382c148a156383b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f31fbe2b067445f1b57538a23eddbe98', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:07.238485 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4d939-c31d-46cc-8ff7-3b22370e9ca0 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:07.238762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4d939-c31d-46cc-8ff7-3b22370e9ca0 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:07.239035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4d939-c31d-46cc-8ff7-3b22370e9ca0 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:07.239694 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4d939-c31d-46cc-8ff7-3b22370e9ca0 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:05:07.240038 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4d939-c31d-46cc-8ff7-3b22370e9ca0 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:07.241873 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4d939-c31d-46cc-8ff7-3b22370e9ca0 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:07.254934 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1284/2555] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 14:05:07.261357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6d39b-c822-4ea9-88ed-cc3e2281890b service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:07.271003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6d39b-c822-4ea9-88ed-cc3e2281890b service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:07.275956 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec4e33-60e4-4348-98f6-e41c7771e38d tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d66a42dde6d45b382c148a156383b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f31fbe2b067445f1b57538a23eddbe98', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:07.276562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec4e33-60e4-4348-98f6-e41c7771e38d tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:07.276844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec4e33-60e4-4348-98f6-e41c7771e38d tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:07.277117 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec4e33-60e4-4348-98f6-e41c7771e38d tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:07.277747 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec4e33-60e4-4348-98f6-e41c7771e38d tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:07.279662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec4e33-60e4-4348-98f6-e41c7771e38d tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:07.312039 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1272/2556] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:05:07 2022] POST /identity/v3/projects => generated 354 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:05:07.316473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4d939-c31d-46cc-8ff7-3b22370e9ca0 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:07.326738 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4d939-c31d-46cc-8ff7-3b22370e9ca0 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:07.386459 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86fa44-6024-42b8-a7a0-fa31a95e046c tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d66a42dde6d45b382c148a156383b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f31fbe2b067445f1b57538a23eddbe98', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:07.387296 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86fa44-6024-42b8-a7a0-fa31a95e046c tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:07.387680 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86fa44-6024-42b8-a7a0-fa31a95e046c tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:07.388050 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86fa44-6024-42b8-a7a0-fa31a95e046c tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:07.389042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86fa44-6024-42b8-a7a0-fa31a95e046c tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:05:07.389486 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86fa44-6024-42b8-a7a0-fa31a95e046c tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:07.392018 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86fa44-6024-42b8-a7a0-fa31a95e046c tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:07.406945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1285/2557] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 14:05:07.445511 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec4e33-60e4-4348-98f6-e41c7771e38d tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:07.457084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec4e33-60e4-4348-98f6-e41c7771e38d tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:07.462447 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5308e23-fff7-4165-8b20-0ec5f8240987 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d66a42dde6d45b382c148a156383b67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f31fbe2b067445f1b57538a23eddbe98', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:07.463145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5308e23-fff7-4165-8b20-0ec5f8240987 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:07.463439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5308e23-fff7-4165-8b20-0ec5f8240987 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:07.463723 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5308e23-fff7-4165-8b20-0ec5f8240987 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] PATH_INFO: `/v3/projects/dd5cf7de14bc4e08919f25ad3ad791fb` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:07.464924 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5308e23-fff7-4165-8b20-0ec5f8240987 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] RBAC: Authorizing `identity:delete_project(project_id=dd5cf7de14bc4e08919f25ad3ad791fb)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:07.466834 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5308e23-fff7-4165-8b20-0ec5f8240987 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:07.515594 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b5308e23-fff7-4165-8b20-0ec5f8240987 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd5cf7de14bc4e08919f25ad3ad791fb', 'request_id': 'req-b5308e23-fff7-4165-8b20-0ec5f8240987'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:07.522266 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b5308e23-fff7-4165-8b20-0ec5f8240987 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd5cf7de14bc4e08919f25ad3ad791fb', 'request_id': 'req-b5308e23-fff7-4165-8b20-0ec5f8240987'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:07.531624 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b5308e23-fff7-4165-8b20-0ec5f8240987 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd5cf7de14bc4e08919f25ad3ad791fb', 'request_id': 'req-b5308e23-fff7-4165-8b20-0ec5f8240987'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:07.543057 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b5308e23-fff7-4165-8b20-0ec5f8240987 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] The token cache is being invalidate because project dd5cf7de14bc4e08919f25ad3ad791fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:05:07.543341 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b5308e23-fff7-4165-8b20-0ec5f8240987 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:07.544032 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b5308e23-fff7-4165-8b20-0ec5f8240987 tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:07.545525 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1273/2558] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:05:07 2022] DELETE /identity/v3/projects/dd5cf7de14bc4e08919f25ad3ad791fb => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:07.767452 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86fa44-6024-42b8-a7a0-fa31a95e046c tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:07.769479 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-6f86fa44-6024-42b8-a7a0-fa31a95e046c tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:07.802396 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=100 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:07.845627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86fa44-6024-42b8-a7a0-fa31a95e046c tempest-VolumeQuotaClassesTest-199424457 tempest-VolumeQuotaClassesTest-199424457-project-admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:07.908075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ac333-268c-4414-b97c-22ce1797a300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:07.908713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ac333-268c-4414-b97c-22ce1797a300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:07.908985 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ac333-268c-4414-b97c-22ce1797a300 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:07.909251 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ac333-268c-4414-b97c-22ce1797a300 admin admin] PATH_INFO: `/v3/users/1e9e058ab3e24488965ac3f4639a7072` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:07.910375 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ac333-268c-4414-b97c-22ce1797a300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e9e058ab3e24488965ac3f4639a7072)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:07.912354 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ac333-268c-4414-b97c-22ce1797a300 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:07.969638 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-3e2ac333-268c-4414-b97c-22ce1797a300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e9e058ab3e24488965ac3f4639a7072'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:07.976524 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-3e2ac333-268c-4414-b97c-22ce1797a300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e9e058ab3e24488965ac3f4639a7072'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:07.977193 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-3e2ac333-268c-4414-b97c-22ce1797a300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e9e058ab3e24488965ac3f4639a7072'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:07.977790 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-3e2ac333-268c-4414-b97c-22ce1797a300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e9e058ab3e24488965ac3f4639a7072'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:07.986398 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-3e2ac333-268c-4414-b97c-22ce1797a300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e9e058ab3e24488965ac3f4639a7072'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:08.003767 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1286/2559] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:05:07 2022] DELETE /identity/v3/users/1e9e058ab3e24488965ac3f4639a7072 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:08.008512 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cda373-0823-4fd5-bb68-0efc26eb70f9 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:08.008881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cda373-0823-4fd5-bb68-0efc26eb70f9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:08.009161 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cda373-0823-4fd5-bb68-0efc26eb70f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:08.030427 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e3cda373-0823-4fd5-bb68-0efc26eb70f9 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:08.031703 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e3cda373-0823-4fd5-bb68-0efc26eb70f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:08.102864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:08.118650 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1274/2560] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:08.128470 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ac333-268c-4414-b97c-22ce1797a300 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:08.130035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3e2ac333-268c-4414-b97c-22ce1797a300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:08.192945 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ac333-268c-4414-b97c-22ce1797a300 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:08.197917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a6839-75fe-42ee-ad16-98788d192784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:08.198533 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a6839-75fe-42ee-ad16-98788d192784 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:08.198811 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a6839-75fe-42ee-ad16-98788d192784 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:08.199077 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a6839-75fe-42ee-ad16-98788d192784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:08.207156 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:08.209723 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a6839-75fe-42ee-ad16-98788d192784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:08.211490 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a6839-75fe-42ee-ad16-98788d192784 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:08.221800 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1287/2561] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:05:08.501090 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cda373-0823-4fd5-bb68-0efc26eb70f9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:08.502607 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e3cda373-0823-4fd5-bb68-0efc26eb70f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:08.515949 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cda373-0823-4fd5-bb68-0efc26eb70f9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:08.520682 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:08.521357 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:08.522145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:08.522420 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] PATH_INFO: `/v3/projects/7fc47386e20a4b518ca8abd60160d7cc` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:08.523834 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc47386e20a4b518ca8abd60160d7cc)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:08.525704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:08.577872 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc47386e20a4b518ca8abd60160d7cc', 'request_id': 'req-885c1735-a5cb-4cb7-b2f3-c66842001b44'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:08.584003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc47386e20a4b518ca8abd60160d7cc', 'request_id': 'req-885c1735-a5cb-4cb7-b2f3-c66842001b44'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:08.594829 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc47386e20a4b518ca8abd60160d7cc', 'request_id': 'req-885c1735-a5cb-4cb7-b2f3-c66842001b44'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:08.605533 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] The token cache is being invalidate because project 7fc47386e20a4b518ca8abd60160d7cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:05:08.605814 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:08.606475 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:08.608084 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1275/2562] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:05:08 2022] DELETE /identity/v3/projects/7fc47386e20a4b518ca8abd60160d7cc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:08.612388 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a6839-75fe-42ee-ad16-98788d192784 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:08.622972 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a6839-75fe-42ee-ad16-98788d192784 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:08.680717 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238e61b-6fdd-45d5-af37-eacdbbd81890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:08.681351 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238e61b-6fdd-45d5-af37-eacdbbd81890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:08.681628 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238e61b-6fdd-45d5-af37-eacdbbd81890 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:08.681899 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238e61b-6fdd-45d5-af37-eacdbbd81890 admin admin] PATH_INFO: `/v3/users/0d66a42dde6d45b382c148a156383b67` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:08.683047 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238e61b-6fdd-45d5-af37-eacdbbd81890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d66a42dde6d45b382c148a156383b67)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:08.685017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238e61b-6fdd-45d5-af37-eacdbbd81890 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:08.737021 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6238e61b-6fdd-45d5-af37-eacdbbd81890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d66a42dde6d45b382c148a156383b67'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:08.743861 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6238e61b-6fdd-45d5-af37-eacdbbd81890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d66a42dde6d45b382c148a156383b67'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:08.744541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6238e61b-6fdd-45d5-af37-eacdbbd81890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d66a42dde6d45b382c148a156383b67'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:08.745138 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6238e61b-6fdd-45d5-af37-eacdbbd81890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d66a42dde6d45b382c148a156383b67'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:08.754689 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-6238e61b-6fdd-45d5-af37-eacdbbd81890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d66a42dde6d45b382c148a156383b67'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:08.772668 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1288/2563] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:05:08 2022] DELETE /identity/v3/users/0d66a42dde6d45b382c148a156383b67 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:09.059610 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:09.061119 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:09.074748 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c1735-a5cb-4cb7-b2f3-c66842001b44 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:09.137047 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7f0f5-b04b-4162-9d79-686ec8cd3812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:09.137704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7f0f5-b04b-4162-9d79-686ec8cd3812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:09.137923 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7f0f5-b04b-4162-9d79-686ec8cd3812 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:09.138269 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7f0f5-b04b-4162-9d79-686ec8cd3812 admin admin] PATH_INFO: `/v3/projects/f31fbe2b067445f1b57538a23eddbe98` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:09.139379 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7f0f5-b04b-4162-9d79-686ec8cd3812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f31fbe2b067445f1b57538a23eddbe98)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:09.141431 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7f0f5-b04b-4162-9d79-686ec8cd3812 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:09.191700 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-89a7f0f5-b04b-4162-9d79-686ec8cd3812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f31fbe2b067445f1b57538a23eddbe98', 'request_id': 'req-89a7f0f5-b04b-4162-9d79-686ec8cd3812'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:09.198796 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-89a7f0f5-b04b-4162-9d79-686ec8cd3812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f31fbe2b067445f1b57538a23eddbe98', 'request_id': 'req-89a7f0f5-b04b-4162-9d79-686ec8cd3812'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:09.207544 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-89a7f0f5-b04b-4162-9d79-686ec8cd3812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f31fbe2b067445f1b57538a23eddbe98', 'request_id': 'req-89a7f0f5-b04b-4162-9d79-686ec8cd3812'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:09.218321 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-89a7f0f5-b04b-4162-9d79-686ec8cd3812 admin admin] The token cache is being invalidate because project f31fbe2b067445f1b57538a23eddbe98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:05:09.219007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-89a7f0f5-b04b-4162-9d79-686ec8cd3812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:09.219636 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-89a7f0f5-b04b-4162-9d79-686ec8cd3812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:09.221161 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1276/2564] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:05:09 2022] DELETE /identity/v3/projects/f31fbe2b067445f1b57538a23eddbe98 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:09.226342 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4325b3-9955-4407-808a-3a54a66ec8aa None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:09.226844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4325b3-9955-4407-808a-3a54a66ec8aa None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:09.226844 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4325b3-9955-4407-808a-3a54a66ec8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:09.247924 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-3f4325b3-9955-4407-808a-3a54a66ec8aa None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:09.249150 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-3f4325b3-9955-4407-808a-3a54a66ec8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:09.313779 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1289/2565] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:09.380464 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce09f9a-9845-49bd-ba81-8a971d7dc4cf None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:09.380778 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce09f9a-9845-49bd-ba81-8a971d7dc4cf None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:09.380928 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce09f9a-9845-49bd-ba81-8a971d7dc4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:09.396071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:09.402146 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6ce09f9a-9845-49bd-ba81-8a971d7dc4cf None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:09.404852 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6ce09f9a-9845-49bd-ba81-8a971d7dc4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:09.413680 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1277/2566] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:09.418988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4325b3-9955-4407-808a-3a54a66ec8aa None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:09.432066 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4325b3-9955-4407-808a-3a54a66ec8aa None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:09.436966 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e5fa0-bb2e-43e4-8cff-2f2e6193e673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:09.437573 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e5fa0-bb2e-43e4-8cff-2f2e6193e673 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:09.437852 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e5fa0-bb2e-43e4-8cff-2f2e6193e673 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:09.438117 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e5fa0-bb2e-43e4-8cff-2f2e6193e673 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:09.438762 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e5fa0-bb2e-43e4-8cff-2f2e6193e673 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:05:09.439059 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e5fa0-bb2e-43e4-8cff-2f2e6193e673 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:09.440712 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e5fa0-bb2e-43e4-8cff-2f2e6193e673 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:09.452445 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1290/2567] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 14:05:09.457479 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce09f9a-9845-49bd-ba81-8a971d7dc4cf None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:09.467610 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce09f9a-9845-49bd-ba81-8a971d7dc4cf None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:09.473593 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b65665-7216-4142-9eaa-5a0939a1d6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:09.474168 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b65665-7216-4142-9eaa-5a0939a1d6ab admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:09.474453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b65665-7216-4142-9eaa-5a0939a1d6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:09.474732 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b65665-7216-4142-9eaa-5a0939a1d6ab admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:09.475360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b65665-7216-4142-9eaa-5a0939a1d6ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:09.477321 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b65665-7216-4142-9eaa-5a0939a1d6ab admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:09.508794 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1278/2568] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:05:09 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:05:09.514832 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e5fa0-bb2e-43e4-8cff-2f2e6193e673 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:09.523162 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:09.530480 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e5fa0-bb2e-43e4-8cff-2f2e6193e673 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:09.598268 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110534f-ff2f-4eed-bf9e-d6ba50d1d813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:09.598890 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110534f-ff2f-4eed-bf9e-d6ba50d1d813 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:09.599163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110534f-ff2f-4eed-bf9e-d6ba50d1d813 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:09.599442 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110534f-ff2f-4eed-bf9e-d6ba50d1d813 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:09.600105 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110534f-ff2f-4eed-bf9e-d6ba50d1d813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:09.602124 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110534f-ff2f-4eed-bf9e-d6ba50d1d813 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:09.628194 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1291/2569] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:05:09 2022] POST /identity/v3/users => generated 284 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:05:09.632932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b65665-7216-4142-9eaa-5a0939a1d6ab admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:09.643774 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b65665-7216-4142-9eaa-5a0939a1d6ab admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:09.648900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873282b-0355-4c21-8196-edfb16ad5739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:09.649446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873282b-0355-4c21-8196-edfb16ad5739 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:09.649720 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873282b-0355-4c21-8196-edfb16ad5739 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:09.649984 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873282b-0355-4c21-8196-edfb16ad5739 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:09.650624 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873282b-0355-4c21-8196-edfb16ad5739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:09.652284 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873282b-0355-4c21-8196-edfb16ad5739 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:09.662693 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1279/2570] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:05:09.667159 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110534f-ff2f-4eed-bf9e-d6ba50d1d813 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:09.677823 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110534f-ff2f-4eed-bf9e-d6ba50d1d813 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:09.683421 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d90ef7-999f-49a4-a68d-b082412a99f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:09.684131 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d90ef7-999f-49a4-a68d-b082412a99f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:09.684404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d90ef7-999f-49a4-a68d-b082412a99f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:09.684673 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d90ef7-999f-49a4-a68d-b082412a99f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:09.685313 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d90ef7-999f-49a4-a68d-b082412a99f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:09.686901 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d90ef7-999f-49a4-a68d-b082412a99f7 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:09.697763 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1292/2571] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:05:09.702781 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873282b-0355-4c21-8196-edfb16ad5739 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:09.719528 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873282b-0355-4c21-8196-edfb16ad5739 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:09.727270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334b4f5-955b-467a-8307-35fd3dade141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:09.727948 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334b4f5-955b-467a-8307-35fd3dade141 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:09.728219 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334b4f5-955b-467a-8307-35fd3dade141 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:09.728480 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334b4f5-955b-467a-8307-35fd3dade141 admin admin] PATH_INFO: `/v3/projects/888d3b333326420385f398ae9b7be346/users/31629246b8154271b08530fee680825a/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:09.744587 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334b4f5-955b-467a-8307-35fd3dade141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=888d3b333326420385f398ae9b7be346, user_id=31629246b8154271b08530fee680825a, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:09.747287 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334b4f5-955b-467a-8307-35fd3dade141 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:09.764775 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1280/2572] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:05:09 2022] PUT /identity/v3/projects/888d3b333326420385f398ae9b7be346/users/31629246b8154271b08530fee680825a/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:09.774986 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d858d2-9de1-4f80-be3c-5f307906b311 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:09.775266 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d858d2-9de1-4f80-be3c-5f307906b311 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:09.775535 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d858d2-9de1-4f80-be3c-5f307906b311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:09.805866 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-05d858d2-9de1-4f80-be3c-5f307906b311 None None] MFA Rules not processed for user `31629246b8154271b08530fee680825a`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:09.807260 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-05d858d2-9de1-4f80-be3c-5f307906b311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:09.880359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1293/2573] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:09.885142 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334b4f5-955b-467a-8307-35fd3dade141 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:09.951193 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334b4f5-955b-467a-8307-35fd3dade141 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:09.956559 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1d9b1-549b-44e2-86fd-bb20b6e89da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:09.957165 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1d9b1-549b-44e2-86fd-bb20b6e89da3 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:09.957444 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1d9b1-549b-44e2-86fd-bb20b6e89da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:09.957715 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1d9b1-549b-44e2-86fd-bb20b6e89da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:09.958343 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d1d9b1-549b-44e2-86fd-bb20b6e89da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:09.960295 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d1d9b1-549b-44e2-86fd-bb20b6e89da3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:09.991007 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1281/2574] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:05:09 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:05:09.996713 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d858d2-9de1-4f80-be3c-5f307906b311 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:10.008817 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d858d2-9de1-4f80-be3c-5f307906b311 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:10.078048 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9bcb3-b477-4792-a95c-a1b4e5e9fe31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:10.078634 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9bcb3-b477-4792-a95c-a1b4e5e9fe31 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:10.078916 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9bcb3-b477-4792-a95c-a1b4e5e9fe31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:10.079195 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9bcb3-b477-4792-a95c-a1b4e5e9fe31 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:10.079829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9bcb3-b477-4792-a95c-a1b4e5e9fe31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:10.081768 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9bcb3-b477-4792-a95c-a1b4e5e9fe31 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:10.100930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:10.106854 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1294/2575] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:05:09 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:05:10.110767 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1d9b1-549b-44e2-86fd-bb20b6e89da3 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:10.121226 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1d9b1-549b-44e2-86fd-bb20b6e89da3 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:10.127083 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a4ae8-1296-4e15-85db-8802c9d1cfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:10.127697 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a4ae8-1296-4e15-85db-8802c9d1cfaa admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:10.128010 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a4ae8-1296-4e15-85db-8802c9d1cfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:10.128278 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a4ae8-1296-4e15-85db-8802c9d1cfaa admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:10.128920 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a4ae8-1296-4e15-85db-8802c9d1cfaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:10.130601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a4ae8-1296-4e15-85db-8802c9d1cfaa admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:10.142333 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1282/2576] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:05:10.146926 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9bcb3-b477-4792-a95c-a1b4e5e9fe31 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:10.157381 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9bcb3-b477-4792-a95c-a1b4e5e9fe31 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:10.162334 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d21313-b6db-4de2-a3c7-00588a7c1371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:10.163002 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d21313-b6db-4de2-a3c7-00588a7c1371 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:10.163333 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d21313-b6db-4de2-a3c7-00588a7c1371 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:10.163621 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d21313-b6db-4de2-a3c7-00588a7c1371 admin admin] PATH_INFO: `/v3/projects/43c53d08adef412b9f44ce5ea9de852c/users/603d82439d284e8f9d709305d6218929/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:10.178863 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d21313-b6db-4de2-a3c7-00588a7c1371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c53d08adef412b9f44ce5ea9de852c, user_id=603d82439d284e8f9d709305d6218929, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:10.181692 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d21313-b6db-4de2-a3c7-00588a7c1371 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:10.198151 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1295/2577] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:05:10 2022] PUT /identity/v3/projects/43c53d08adef412b9f44ce5ea9de852c/users/603d82439d284e8f9d709305d6218929/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:10.202464 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a4ae8-1296-4e15-85db-8802c9d1cfaa admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:10.213663 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a4ae8-1296-4e15-85db-8802c9d1cfaa admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:10.286959 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d51fba-0387-4a3e-b3a0-4f5f7d307e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:10.287562 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d51fba-0387-4a3e-b3a0-4f5f7d307e95 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:10.287833 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d51fba-0387-4a3e-b3a0-4f5f7d307e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:10.288129 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d51fba-0387-4a3e-b3a0-4f5f7d307e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:10.288734 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d51fba-0387-4a3e-b3a0-4f5f7d307e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:10.290805 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d51fba-0387-4a3e-b3a0-4f5f7d307e95 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:10.301991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1283/2578] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:05:10.306518 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d21313-b6db-4de2-a3c7-00588a7c1371 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:10.317423 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d21313-b6db-4de2-a3c7-00588a7c1371 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:10.322438 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb100a-e06d-4ec9-a364-7794a484f911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:10.323091 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb100a-e06d-4ec9-a364-7794a484f911 admin admin] REQUEST_METHOD: `PUT` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:10.323359 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb100a-e06d-4ec9-a364-7794a484f911 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:10.323625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb100a-e06d-4ec9-a364-7794a484f911 admin admin] PATH_INFO: `/v3/projects/43c53d08adef412b9f44ce5ea9de852c/users/603d82439d284e8f9d709305d6218929/roles/57740dc2ebe1429294f20dd51592b82a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:10.326806 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb100a-e06d-4ec9-a364-7794a484f911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c53d08adef412b9f44ce5ea9de852c, user_id=603d82439d284e8f9d709305d6218929, role_id=57740dc2ebe1429294f20dd51592b82a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:10.331170 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb100a-e06d-4ec9-a364-7794a484f911 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:10.345733 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1296/2579] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:05:10 2022] PUT /identity/v3/projects/43c53d08adef412b9f44ce5ea9de852c/users/603d82439d284e8f9d709305d6218929/roles/57740dc2ebe1429294f20dd51592b82a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:10.357957 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85577a8b-dff9-495f-bea3-ffbafa2f54a8 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:10.358221 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85577a8b-dff9-495f-bea3-ffbafa2f54a8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:10.358454 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85577a8b-dff9-495f-bea3-ffbafa2f54a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:10.385826 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:10.389382 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-85577a8b-dff9-495f-bea3-ffbafa2f54a8 None None] MFA Rules not processed for user `603d82439d284e8f9d709305d6218929`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:10.391613 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-85577a8b-dff9-495f-bea3-ffbafa2f54a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:10.471003 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1284/2580] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:10.486255 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb100a-e06d-4ec9-a364-7794a484f911 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:10.488498 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-5eeb100a-e06d-4ec9-a364-7794a484f911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:10.551417 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb100a-e06d-4ec9-a364-7794a484f911 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:10.555513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9fca2-0674-4577-9f56-fb853dc8f255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:10.556161 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9fca2-0674-4577-9f56-fb853dc8f255 service cinder] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:10.556437 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9fca2-0674-4577-9f56-fb853dc8f255 service cinder] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:10.556704 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9fca2-0674-4577-9f56-fb853dc8f255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:10.566247 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9fca2-0674-4577-9f56-fb853dc8f255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:10.567680 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9fca2-0674-4577-9f56-fb853dc8f255 service cinder] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:10.577204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1297/2581] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:05:10 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:05:10.674530 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85577a8b-dff9-495f-bea3-ffbafa2f54a8 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:10.684885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85577a8b-dff9-495f-bea3-ffbafa2f54a8 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:10.689378 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0e4cf-ee72-4746-b3b6-ff1d433c5e83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:10.689988 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0e4cf-ee72-4746-b3b6-ff1d433c5e83 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:10.690259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0e4cf-ee72-4746-b3b6-ff1d433c5e83 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:10.690525 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0e4cf-ee72-4746-b3b6-ff1d433c5e83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:10.749793 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0e4cf-ee72-4746-b3b6-ff1d433c5e83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:10.751162 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0e4cf-ee72-4746-b3b6-ff1d433c5e83 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:10.760547 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1285/2582] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:05:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:05:15.648658 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9fca2-0674-4577-9f56-fb853dc8f255 service cinder] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:15.716822 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9fca2-0674-4577-9f56-fb853dc8f255 service cinder] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:15.722112 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859bb6e-0168-4d58-8b04-0d7ae5f3a41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:15.722747 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859bb6e-0168-4d58-8b04-0d7ae5f3a41f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:15.723023 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859bb6e-0168-4d58-8b04-0d7ae5f3a41f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:15.723292 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859bb6e-0168-4d58-8b04-0d7ae5f3a41f admin admin] PATH_INFO: `/v3/users/31629246b8154271b08530fee680825a` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:15.724473 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859bb6e-0168-4d58-8b04-0d7ae5f3a41f admin admin] RBAC: Authorizing `identity:delete_user(user_id=31629246b8154271b08530fee680825a)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:15.726378 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859bb6e-0168-4d58-8b04-0d7ae5f3a41f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:15.784622 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-0859bb6e-0168-4d58-8b04-0d7ae5f3a41f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31629246b8154271b08530fee680825a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:15.792629 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-0859bb6e-0168-4d58-8b04-0d7ae5f3a41f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31629246b8154271b08530fee680825a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:15.793376 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-0859bb6e-0168-4d58-8b04-0d7ae5f3a41f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31629246b8154271b08530fee680825a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:15.794000 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-0859bb6e-0168-4d58-8b04-0d7ae5f3a41f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31629246b8154271b08530fee680825a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:15.803917 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-0859bb6e-0168-4d58-8b04-0d7ae5f3a41f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31629246b8154271b08530fee680825a'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:15.809587 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:15.828320 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1298/2583] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:05:15 2022] DELETE /identity/v3/users/31629246b8154271b08530fee680825a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:15.832875 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88380041-e4e0-4691-9354-62cd979967e7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:15.833388 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88380041-e4e0-4691-9354-62cd979967e7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:15.833502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88380041-e4e0-4691-9354-62cd979967e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:15.854576 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-88380041-e4e0-4691-9354-62cd979967e7 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:15.856111 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-88380041-e4e0-4691-9354-62cd979967e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:15.925605 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1286/2584] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:15.935641 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859bb6e-0168-4d58-8b04-0d7ae5f3a41f admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:15.937424 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-0859bb6e-0168-4d58-8b04-0d7ae5f3a41f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:16.001017 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859bb6e-0168-4d58-8b04-0d7ae5f3a41f admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:16.004765 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ceba9-e42f-4169-8bca-bdc154696c50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:16.005352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ceba9-e42f-4169-8bca-bdc154696c50 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:16.005653 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ceba9-e42f-4169-8bca-bdc154696c50 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:16.005925 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ceba9-e42f-4169-8bca-bdc154696c50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:16.015175 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ceba9-e42f-4169-8bca-bdc154696c50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:16.016654 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ceba9-e42f-4169-8bca-bdc154696c50 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:16.026035 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1299/2585] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:05:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:05:16.329448 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88380041-e4e0-4691-9354-62cd979967e7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:16.330993 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-88380041-e4e0-4691-9354-62cd979967e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:16.346141 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88380041-e4e0-4691-9354-62cd979967e7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:16.352367 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:16.353070 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:16.353300 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:16.353732 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] PATH_INFO: `/v3/projects/888d3b333326420385f398ae9b7be346` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:16.355138 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=888d3b333326420385f398ae9b7be346)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:16.357248 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:16.396918 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:16.410620 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '888d3b333326420385f398ae9b7be346', 'request_id': 'req-912ce173-2ddb-4074-ae24-22e3783e7826'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:16.417334 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '888d3b333326420385f398ae9b7be346', 'request_id': 'req-912ce173-2ddb-4074-ae24-22e3783e7826'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:16.426434 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '888d3b333326420385f398ae9b7be346', 'request_id': 'req-912ce173-2ddb-4074-ae24-22e3783e7826'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:16.437601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] The token cache is being invalidate because project 888d3b333326420385f398ae9b7be346 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:05:16.437885 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:16.438553 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:16.440140 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1287/2586] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:05:16 2022] DELETE /identity/v3/projects/888d3b333326420385f398ae9b7be346 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:16.444455 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ceba9-e42f-4169-8bca-bdc154696c50 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:16.454625 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ceba9-e42f-4169-8bca-bdc154696c50 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:16.515930 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a05a67-1e45-4e82-b9d0-eb25c5924b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:16.516597 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a05a67-1e45-4e82-b9d0-eb25c5924b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:16.516883 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a05a67-1e45-4e82-b9d0-eb25c5924b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:16.517160 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a05a67-1e45-4e82-b9d0-eb25c5924b21 admin admin] PATH_INFO: `/v3/users/603d82439d284e8f9d709305d6218929` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:16.518453 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a05a67-1e45-4e82-b9d0-eb25c5924b21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=603d82439d284e8f9d709305d6218929)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:16.520482 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a05a67-1e45-4e82-b9d0-eb25c5924b21 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:16.583547 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f8a05a67-1e45-4e82-b9d0-eb25c5924b21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '603d82439d284e8f9d709305d6218929'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:16.594908 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f8a05a67-1e45-4e82-b9d0-eb25c5924b21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '603d82439d284e8f9d709305d6218929'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:16.596163 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f8a05a67-1e45-4e82-b9d0-eb25c5924b21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '603d82439d284e8f9d709305d6218929'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:16.597273 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f8a05a67-1e45-4e82-b9d0-eb25c5924b21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '603d82439d284e8f9d709305d6218929'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:16.609976 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-f8a05a67-1e45-4e82-b9d0-eb25c5924b21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '603d82439d284e8f9d709305d6218929'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:16.631775 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1300/2587] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:05:16 2022] DELETE /identity/v3/users/603d82439d284e8f9d709305d6218929 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:16.923027 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:16.924625 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:16.938041 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ce173-2ddb-4074-ae24-22e3783e7826 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:16.998721 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d63268-c25d-422e-bab5-3870e0fc835b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:16.999361 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d63268-c25d-422e-bab5-3870e0fc835b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:16.999639 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d63268-c25d-422e-bab5-3870e0fc835b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:16.999916 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d63268-c25d-422e-bab5-3870e0fc835b admin admin] PATH_INFO: `/v3/projects/43c53d08adef412b9f44ce5ea9de852c` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:17.001086 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d63268-c25d-422e-bab5-3870e0fc835b admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c53d08adef412b9f44ce5ea9de852c)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:17.003002 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d63268-c25d-422e-bab5-3870e0fc835b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:17.057680 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-35d63268-c25d-422e-bab5-3870e0fc835b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c53d08adef412b9f44ce5ea9de852c', 'request_id': 'req-35d63268-c25d-422e-bab5-3870e0fc835b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:17.064270 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-35d63268-c25d-422e-bab5-3870e0fc835b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c53d08adef412b9f44ce5ea9de852c', 'request_id': 'req-35d63268-c25d-422e-bab5-3870e0fc835b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:17.076732 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-35d63268-c25d-422e-bab5-3870e0fc835b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c53d08adef412b9f44ce5ea9de852c', 'request_id': 'req-35d63268-c25d-422e-bab5-3870e0fc835b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:17.091900 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-35d63268-c25d-422e-bab5-3870e0fc835b admin admin] The token cache is being invalidate because project 43c53d08adef412b9f44ce5ea9de852c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:05:17.092216 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-35d63268-c25d-422e-bab5-3870e0fc835b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:17.092878 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-35d63268-c25d-422e-bab5-3870e0fc835b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:17.094807 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1288/2588] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:05:16 2022] DELETE /identity/v3/projects/43c53d08adef412b9f44ce5ea9de852c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 15 14:05:17.099954 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f9a44-f536-4d9c-aa88-38f07b416911 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:17.100425 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f9a44-f536-4d9c-aa88-38f07b416911 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:17.100541 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f9a44-f536-4d9c-aa88-38f07b416911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:17.114738 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:17.121596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-679f9a44-f536-4d9c-aa88-38f07b416911 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:17.123561 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-679f9a44-f536-4d9c-aa88-38f07b416911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:17.186404 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1301/2589] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:17.198502 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2729866-79a0-471f-9021-a5f57f905c02 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:17.198786 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2729866-79a0-471f-9021-a5f57f905c02 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:17.199057 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2729866-79a0-471f-9021-a5f57f905c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:17.220245 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c2729866-79a0-471f-9021-a5f57f905c02 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:17.221468 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c2729866-79a0-471f-9021-a5f57f905c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:17.231188 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1289/2590] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:17.235722 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f9a44-f536-4d9c-aa88-38f07b416911 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:17.246061 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f9a44-f536-4d9c-aa88-38f07b416911 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:17.251583 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d6348-fc79-4aa1-8ea0-5d9f5801c480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:17.252223 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d6348-fc79-4aa1-8ea0-5d9f5801c480 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:17.252497 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d6348-fc79-4aa1-8ea0-5d9f5801c480 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:17.252782 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d6348-fc79-4aa1-8ea0-5d9f5801c480 admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:17.253424 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d6348-fc79-4aa1-8ea0-5d9f5801c480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:05:17.253724 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d6348-fc79-4aa1-8ea0-5d9f5801c480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:17.255315 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d6348-fc79-4aa1-8ea0-5d9f5801c480 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:17.267785 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1302/2591] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 15 14:05:17.272453 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2729866-79a0-471f-9021-a5f57f905c02 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:17.281993 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2729866-79a0-471f-9021-a5f57f905c02 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:17.288017 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef39e1-bba6-4ad2-a536-9d4b999c9849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:17.288644 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef39e1-bba6-4ad2-a536-9d4b999c9849 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:17.288915 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef39e1-bba6-4ad2-a536-9d4b999c9849 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:17.289178 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef39e1-bba6-4ad2-a536-9d4b999c9849 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:17.289824 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef39e1-bba6-4ad2-a536-9d4b999c9849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:17.291914 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef39e1-bba6-4ad2-a536-9d4b999c9849 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:17.325305 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1290/2592] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:05:17 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:05:17.329829 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d6348-fc79-4aa1-8ea0-5d9f5801c480 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:17.342361 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d6348-fc79-4aa1-8ea0-5d9f5801c480 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:17.404192 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3f322-1308-439f-a25f-e27946ea27de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:17.404805 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3f322-1308-439f-a25f-e27946ea27de admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:17.405085 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3f322-1308-439f-a25f-e27946ea27de admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:17.405365 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3f322-1308-439f-a25f-e27946ea27de admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:17.406024 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3f322-1308-439f-a25f-e27946ea27de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:17.408012 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f3f322-1308-439f-a25f-e27946ea27de admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:17.432128 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1303/2593] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:05:17 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 15 14:05:17.436654 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef39e1-bba6-4ad2-a536-9d4b999c9849 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:17.447791 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef39e1-bba6-4ad2-a536-9d4b999c9849 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:17.452578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810366c5-2bc5-4286-ab59-93bcb10a5ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:17.453183 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810366c5-2bc5-4286-ab59-93bcb10a5ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:17.453459 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810366c5-2bc5-4286-ab59-93bcb10a5ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:17.453727 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810366c5-2bc5-4286-ab59-93bcb10a5ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:17.454392 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810366c5-2bc5-4286-ab59-93bcb10a5ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:17.456033 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810366c5-2bc5-4286-ab59-93bcb10a5ff0 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:17.466481 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1291/2594] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:05:17.471008 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3f322-1308-439f-a25f-e27946ea27de admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:17.481476 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3f322-1308-439f-a25f-e27946ea27de admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:17.486585 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d8b3f-18a8-4458-9011-4585563768e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:17.487181 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d8b3f-18a8-4458-9011-4585563768e7 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:17.487477 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d8b3f-18a8-4458-9011-4585563768e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:17.487750 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d8b3f-18a8-4458-9011-4585563768e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:17.488415 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d8b3f-18a8-4458-9011-4585563768e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:17.489988 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d8b3f-18a8-4458-9011-4585563768e7 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:17.501191 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1304/2595] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:05:17.507649 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810366c5-2bc5-4286-ab59-93bcb10a5ff0 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:17.522235 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810366c5-2bc5-4286-ab59-93bcb10a5ff0 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:17.528844 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac662c8-4d72-439d-a827-c096545063d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:17.529499 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac662c8-4d72-439d-a827-c096545063d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:17.529764 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac662c8-4d72-439d-a827-c096545063d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:17.530030 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac662c8-4d72-439d-a827-c096545063d6 admin admin] PATH_INFO: `/v3/projects/2e2db083845a469f8e6e008c00971f60/users/a7d8bdee9d6b4a9ca07c561bd8cea39e/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:17.538439 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:17.547215 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac662c8-4d72-439d-a827-c096545063d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2db083845a469f8e6e008c00971f60, user_id=a7d8bdee9d6b4a9ca07c561bd8cea39e, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:17.550238 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac662c8-4d72-439d-a827-c096545063d6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:17.566260 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1292/2596] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:05:17 2022] PUT /identity/v3/projects/2e2db083845a469f8e6e008c00971f60/users/a7d8bdee9d6b4a9ca07c561bd8cea39e/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:17.576870 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf38388-a338-4274-999e-75fbdc5fe891 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:17.577239 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf38388-a338-4274-999e-75fbdc5fe891 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:17.577570 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf38388-a338-4274-999e-75fbdc5fe891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:17.606596 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-bbf38388-a338-4274-999e-75fbdc5fe891 None None] MFA Rules not processed for user `a7d8bdee9d6b4a9ca07c561bd8cea39e`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:17.607820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-bbf38388-a338-4274-999e-75fbdc5fe891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:17.681302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1305/2597] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:17.692963 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac662c8-4d72-439d-a827-c096545063d6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:17.694665 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9ac662c8-4d72-439d-a827-c096545063d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:17.752615 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac662c8-4d72-439d-a827-c096545063d6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:17.756881 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c1281-a019-4bc9-abf5-19caef4f1e48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:17.757490 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c1281-a019-4bc9-abf5-19caef4f1e48 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:17.757761 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c1281-a019-4bc9-abf5-19caef4f1e48 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:17.758006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c1281-a019-4bc9-abf5-19caef4f1e48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:17.767122 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c1281-a019-4bc9-abf5-19caef4f1e48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:17.768592 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c1281-a019-4bc9-abf5-19caef4f1e48 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:17.778020 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1293/2598] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:05:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:05:27.678294 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:27.773355 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:39.768019 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf38388-a338-4274-999e-75fbdc5fe891 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:39.838878 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf38388-a338-4274-999e-75fbdc5fe891 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:39.844944 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e3adc-1209-4e1b-987c-7eb8d8f122ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:39.845576 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e3adc-1209-4e1b-987c-7eb8d8f122ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:39.845851 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e3adc-1209-4e1b-987c-7eb8d8f122ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:39.846114 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e3adc-1209-4e1b-987c-7eb8d8f122ef admin admin] PATH_INFO: `/v3/users/a7d8bdee9d6b4a9ca07c561bd8cea39e` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:39.847346 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e3adc-1209-4e1b-987c-7eb8d8f122ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7d8bdee9d6b4a9ca07c561bd8cea39e)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:39.849302 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e3adc-1209-4e1b-987c-7eb8d8f122ef admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:39.901711 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-931e3adc-1209-4e1b-987c-7eb8d8f122ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7d8bdee9d6b4a9ca07c561bd8cea39e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:39.908856 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-931e3adc-1209-4e1b-987c-7eb8d8f122ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7d8bdee9d6b4a9ca07c561bd8cea39e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:39.909548 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-931e3adc-1209-4e1b-987c-7eb8d8f122ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7d8bdee9d6b4a9ca07c561bd8cea39e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:39.910184 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-931e3adc-1209-4e1b-987c-7eb8d8f122ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7d8bdee9d6b4a9ca07c561bd8cea39e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:39.919520 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-931e3adc-1209-4e1b-987c-7eb8d8f122ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7d8bdee9d6b4a9ca07c561bd8cea39e'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:39.937910 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1306/2599] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:05:39 2022] DELETE /identity/v3/users/a7d8bdee9d6b4a9ca07c561bd8cea39e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:39.942428 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cab28b-7baa-4a17-a165-4086777d8eae None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:39.942936 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cab28b-7baa-4a17-a165-4086777d8eae None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:39.943053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cab28b-7baa-4a17-a165-4086777d8eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:39.968432 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b3cab28b-7baa-4a17-a165-4086777d8eae None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:39.970147 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b3cab28b-7baa-4a17-a165-4086777d8eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:40.034200 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1294/2600] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:40.046322 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e3adc-1209-4e1b-987c-7eb8d8f122ef admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:40.049026 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-931e3adc-1209-4e1b-987c-7eb8d8f122ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:40.121681 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e3adc-1209-4e1b-987c-7eb8d8f122ef admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:40.127380 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd4730-b8d7-4d99-8ab2-ab0995acce6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:40.128371 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd4730-b8d7-4d99-8ab2-ab0995acce6c service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:40.128799 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd4730-b8d7-4d99-8ab2-ab0995acce6c service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:40.129193 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd4730-b8d7-4d99-8ab2-ab0995acce6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:40.143321 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd4730-b8d7-4d99-8ab2-ab0995acce6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:40.145245 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd4730-b8d7-4d99-8ab2-ab0995acce6c service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:40.158042 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1307/2601] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:05:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:05:40.469608 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cab28b-7baa-4a17-a165-4086777d8eae None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:40.471972 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b3cab28b-7baa-4a17-a165-4086777d8eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:40.487634 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cab28b-7baa-4a17-a165-4086777d8eae None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:40.492492 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602af1d-1894-4e82-b245-88b46815808d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:40.493127 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602af1d-1894-4e82-b245-88b46815808d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:40.493412 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602af1d-1894-4e82-b245-88b46815808d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:40.493662 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602af1d-1894-4e82-b245-88b46815808d admin admin] PATH_INFO: `/v3/projects/2e2db083845a469f8e6e008c00971f60` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:40.494825 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602af1d-1894-4e82-b245-88b46815808d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e2db083845a469f8e6e008c00971f60)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:40.496793 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602af1d-1894-4e82-b245-88b46815808d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:40.548391 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2602af1d-1894-4e82-b245-88b46815808d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2db083845a469f8e6e008c00971f60', 'request_id': 'req-2602af1d-1894-4e82-b245-88b46815808d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:40.557091 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2602af1d-1894-4e82-b245-88b46815808d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2db083845a469f8e6e008c00971f60', 'request_id': 'req-2602af1d-1894-4e82-b245-88b46815808d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:40.570071 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2602af1d-1894-4e82-b245-88b46815808d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2db083845a469f8e6e008c00971f60', 'request_id': 'req-2602af1d-1894-4e82-b245-88b46815808d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:40.581773 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2602af1d-1894-4e82-b245-88b46815808d admin admin] The token cache is being invalidate because project 2e2db083845a469f8e6e008c00971f60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:05:40.582059 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2602af1d-1894-4e82-b245-88b46815808d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:40.582689 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2602af1d-1894-4e82-b245-88b46815808d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:05:40.584224 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1295/2602] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:05:40 2022] DELETE /identity/v3/projects/2e2db083845a469f8e6e008c00971f60 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:40.590352 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3af37d-201f-4f08-a4e9-21725f99a03a None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:40.590634 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3af37d-201f-4f08-a4e9-21725f99a03a None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:40.590904 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3af37d-201f-4f08-a4e9-21725f99a03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:40.611643 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-fb3af37d-201f-4f08-a4e9-21725f99a03a None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:40.612901 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-fb3af37d-201f-4f08-a4e9-21725f99a03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:40.634486 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:40.686736 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1308/2603] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:40.694599 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53266bf3-fc7f-4a58-a93e-dd0e9e7a3790 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:40.694864 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53266bf3-fc7f-4a58-a93e-dd0e9e7a3790 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:40.695140 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53266bf3-fc7f-4a58-a93e-dd0e9e7a3790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:40.716185 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-53266bf3-fc7f-4a58-a93e-dd0e9e7a3790 None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:40.717433 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-53266bf3-fc7f-4a58-a93e-dd0e9e7a3790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:40.727578 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1296/2604] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:40.732162 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3af37d-201f-4f08-a4e9-21725f99a03a None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:40.742339 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3af37d-201f-4f08-a4e9-21725f99a03a None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:40.748204 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28827707-a4e4-436d-96aa-b82d3af7312f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:40.748838 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28827707-a4e4-436d-96aa-b82d3af7312f admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:40.749112 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28827707-a4e4-436d-96aa-b82d3af7312f admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:40.749387 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28827707-a4e4-436d-96aa-b82d3af7312f admin admin] PATH_INFO: `/v3/domains` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:40.750047 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28827707-a4e4-436d-96aa-b82d3af7312f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 15 14:05:40.750351 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28827707-a4e4-436d-96aa-b82d3af7312f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:40.752013 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28827707-a4e4-436d-96aa-b82d3af7312f admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:40.765363 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1309/2605] 10.222.0.36 () {58 vars in 1153 bytes} [Sat Oct 15 14:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 15 14:05:40.771298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53266bf3-fc7f-4a58-a93e-dd0e9e7a3790 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:40.781442 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53266bf3-fc7f-4a58-a93e-dd0e9e7a3790 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:40.787207 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89e135-99b0-4931-8982-317a00474c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:40.787842 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89e135-99b0-4931-8982-317a00474c18 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:40.788152 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89e135-99b0-4931-8982-317a00474c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:40.788422 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89e135-99b0-4931-8982-317a00474c18 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:40.789051 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89e135-99b0-4931-8982-317a00474c18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:40.790958 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89e135-99b0-4931-8982-317a00474c18 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:40.828298 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1297/2606] 10.222.0.36 () {60 vars in 1153 bytes} [Sat Oct 15 14:05:40 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:05:40.829465 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28827707-a4e4-436d-96aa-b82d3af7312f admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:40.840197 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28827707-a4e4-436d-96aa-b82d3af7312f admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:40.901731 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea497c4-af19-49ae-aa19-186b93a609f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:40.902349 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea497c4-af19-49ae-aa19-186b93a609f1 admin admin] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:40.902627 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea497c4-af19-49ae-aa19-186b93a609f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:40.902900 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea497c4-af19-49ae-aa19-186b93a609f1 admin admin] PATH_INFO: `/v3/users` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:40.903553 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea497c4-af19-49ae-aa19-186b93a609f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:40.905512 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea497c4-af19-49ae-aa19-186b93a609f1 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:40.928820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1310/2607] 10.222.0.36 () {60 vars in 1144 bytes} [Sat Oct 15 14:05:40 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 15 14:05:40.933209 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89e135-99b0-4931-8982-317a00474c18 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:40.943507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89e135-99b0-4931-8982-317a00474c18 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:40.949082 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1cca7-15eb-49a2-96b7-1caeb5471c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:40.949714 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1cca7-15eb-49a2-96b7-1caeb5471c3c admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:40.949991 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1cca7-15eb-49a2-96b7-1caeb5471c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:40.950289 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1cca7-15eb-49a2-96b7-1caeb5471c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:40.950932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1cca7-15eb-49a2-96b7-1caeb5471c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:40.952595 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1cca7-15eb-49a2-96b7-1caeb5471c3c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:40.962858 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1298/2608] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 15 14:05:40.969810 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea497c4-af19-49ae-aa19-186b93a609f1 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:40.985341 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea497c4-af19-49ae-aa19-186b93a609f1 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:40.993770 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d2c68-3dd6-4fa7-a77b-72b8ddc16132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:40.994655 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d2c68-3dd6-4fa7-a77b-72b8ddc16132 admin admin] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:40.995227 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d2c68-3dd6-4fa7-a77b-72b8ddc16132 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:40.995715 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d2c68-3dd6-4fa7-a77b-72b8ddc16132 admin admin] PATH_INFO: `/v3/roles` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:40.996486 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d2c68-3dd6-4fa7-a77b-72b8ddc16132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:40.998083 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d2c68-3dd6-4fa7-a77b-72b8ddc16132 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:41.013411 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1311/2609] 10.222.0.36 () {58 vars in 1122 bytes} [Sat Oct 15 14:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 15 14:05:41.019862 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1cca7-15eb-49a2-96b7-1caeb5471c3c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:41.032021 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1cca7-15eb-49a2-96b7-1caeb5471c3c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:41.037145 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6500c4-1100-4922-8551-a767a585f716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:41.037631 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6500c4-1100-4922-8551-a767a585f716 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:41.037906 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6500c4-1100-4922-8551-a767a585f716 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:41.038176 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6500c4-1100-4922-8551-a767a585f716 admin admin] PATH_INFO: `/v3/projects/e21a4ed48a484686873dcebe96a0c840/users/045b7e280599412da65804cb517a4e0f/roles/ad5c2cf379e34fdcb85aa27f334e3fc0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:41.053040 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6500c4-1100-4922-8551-a767a585f716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e21a4ed48a484686873dcebe96a0c840, user_id=045b7e280599412da65804cb517a4e0f, role_id=ad5c2cf379e34fdcb85aa27f334e3fc0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:41.055706 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6500c4-1100-4922-8551-a767a585f716 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:41.072932 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1299/2610] 10.222.0.36 () {60 vars in 1483 bytes} [Sat Oct 15 14:05:41 2022] PUT /identity/v3/projects/e21a4ed48a484686873dcebe96a0c840/users/045b7e280599412da65804cb517a4e0f/roles/ad5c2cf379e34fdcb85aa27f334e3fc0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:05:41.083370 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1270d1-8803-4555-ac27-736594a99301 None None] REQUEST_METHOD: `POST` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:41.083649 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1270d1-8803-4555-ac27-736594a99301 None None] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:41.083941 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1270d1-8803-4555-ac27-736594a99301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:41.113854 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.auth.core [None req-2d1270d1-8803-4555-ac27-736594a99301 None None] MFA Rules not processed for user `045b7e280599412da65804cb517a4e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=87915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:05:41.115087 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2d1270d1-8803-4555-ac27-736594a99301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:41.191632 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1312/2611] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:05:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:05:41.203446 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6500c4-1100-4922-8551-a767a585f716 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:05:41.205601 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cf6500c4-1100-4922-8551-a767a585f716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:05:41.229534 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:41.278848 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6500c4-1100-4922-8551-a767a585f716 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:05:41.284354 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc999759-98a3-4e9b-8495-88dba8f1bbb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c96e9f796814f1e8079f3e16073c2eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:05:41.284989 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc999759-98a3-4e9b-8495-88dba8f1bbb0 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:05:41.285259 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc999759-98a3-4e9b-8495-88dba8f1bbb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:05:41.285524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc999759-98a3-4e9b-8495-88dba8f1bbb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:05:41.294775 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc999759-98a3-4e9b-8495-88dba8f1bbb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:05:41.296244 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc999759-98a3-4e9b-8495-88dba8f1bbb0 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:05:41.305610 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1300/2612] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:05:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:05:51.188471 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:05:51.301238 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=17 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:06:01.925848 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1270d1-8803-4555-ac27-736594a99301 None None] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:06:01.995990 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1270d1-8803-4555-ac27-736594a99301 None None] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:06:02.000911 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51c1c4-d9e9-4fe8-840e-52fdbd589eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:06:02.001545 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51c1c4-d9e9-4fe8-840e-52fdbd589eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:06:02.001824 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51c1c4-d9e9-4fe8-840e-52fdbd589eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:06:02.002096 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51c1c4-d9e9-4fe8-840e-52fdbd589eb7 admin admin] PATH_INFO: `/v3/users/045b7e280599412da65804cb517a4e0f` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:06:02.003284 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51c1c4-d9e9-4fe8-840e-52fdbd589eb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=045b7e280599412da65804cb517a4e0f)` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:06:02.005229 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51c1c4-d9e9-4fe8-840e-52fdbd589eb7 admin admin] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:06:02.059513 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-dd51c1c4-d9e9-4fe8-840e-52fdbd589eb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '045b7e280599412da65804cb517a4e0f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:06:02.066897 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-dd51c1c4-d9e9-4fe8-840e-52fdbd589eb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '045b7e280599412da65804cb517a4e0f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:06:02.067601 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-dd51c1c4-d9e9-4fe8-840e-52fdbd589eb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '045b7e280599412da65804cb517a4e0f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:06:02.068271 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-dd51c1c4-d9e9-4fe8-840e-52fdbd589eb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '045b7e280599412da65804cb517a4e0f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:06:02.078205 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.notifications [None req-dd51c1c4-d9e9-4fe8-840e-52fdbd589eb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '045b7e280599412da65804cb517a4e0f'} {{(pid=87915) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:06:02.095536 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1313/2613] 10.222.0.36 () {58 vars in 1224 bytes} [Sat Oct 15 14:06:01 2022] DELETE /identity/v3/users/045b7e280599412da65804cb517a4e0f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:06:02.100151 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef070e-c691-408b-be87-ba26c3280e5e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:06:02.100668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef070e-c691-408b-be87-ba26c3280e5e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:06:02.100668 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef070e-c691-408b-be87-ba26c3280e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:06:02.120006 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d7ef070e-c691-408b-be87-ba26c3280e5e None None] MFA Rules not processed for user `d6c4e60be4154870948fd9be3f7e4091`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 15 14:06:02.121197 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d7ef070e-c691-408b-be87-ba26c3280e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:06:02.183772 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1301/2614] 10.222.0.36 () {58 vars in 958 bytes} [Sat Oct 15 14:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 15 14:06:02.193656 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51c1c4-d9e9-4fe8-840e-52fdbd589eb7 admin admin] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:06:02.195167 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-dd51c1c4-d9e9-4fe8-840e-52fdbd589eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:06:02.255074 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51c1c4-d9e9-4fe8-840e-52fdbd589eb7 admin admin] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:06:02.261921 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efa6b0-14fd-4730-99ff-b08006469020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90985ec8c43a4d7895942731dc1d002c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:06:02.262820 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efa6b0-14fd-4730-99ff-b08006469020 service neutron] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:06:02.263232 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efa6b0-14fd-4730-99ff-b08006469020 service neutron] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:06:02.263685 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efa6b0-14fd-4730-99ff-b08006469020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:06:02.274850 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efa6b0-14fd-4730-99ff-b08006469020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:06:02.276285 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efa6b0-14fd-4730-99ff-b08006469020 service neutron] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:06:02.286068 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1314/2615] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:06:02.565739 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef070e-c691-408b-be87-ba26c3280e5e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:06:02.567416 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d7ef070e-c691-408b-be87-ba26c3280e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:06:02.579845 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef070e-c691-408b-be87-ba26c3280e5e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:06:02.587524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73632501-6eb7-4311-88aa-bd5300956afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6c4e60be4154870948fd9be3f7e4091', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '443104cd704b44c6867e1bc065e90d39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:06:02.588338 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73632501-6eb7-4311-88aa-bd5300956afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:06:02.588726 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73632501-6eb7-4311-88aa-bd5300956afb admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:06:02.589057 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73632501-6eb7-4311-88aa-bd5300956afb admin admin] PATH_INFO: `/v3/projects/e21a4ed48a484686873dcebe96a0c840` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:06:02.590310 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73632501-6eb7-4311-88aa-bd5300956afb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e21a4ed48a484686873dcebe96a0c840)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:06:02.592690 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73632501-6eb7-4311-88aa-bd5300956afb admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:06:02.642053 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73632501-6eb7-4311-88aa-bd5300956afb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e21a4ed48a484686873dcebe96a0c840', 'request_id': 'req-73632501-6eb7-4311-88aa-bd5300956afb'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:06:02.648524 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73632501-6eb7-4311-88aa-bd5300956afb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e21a4ed48a484686873dcebe96a0c840', 'request_id': 'req-73632501-6eb7-4311-88aa-bd5300956afb'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:06:02.658704 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73632501-6eb7-4311-88aa-bd5300956afb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e21a4ed48a484686873dcebe96a0c840', 'request_id': 'req-73632501-6eb7-4311-88aa-bd5300956afb'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:06:02.669076 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73632501-6eb7-4311-88aa-bd5300956afb admin admin] The token cache is being invalidate because project e21a4ed48a484686873dcebe96a0c840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 15 14:06:02.669360 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73632501-6eb7-4311-88aa-bd5300956afb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:06:02.669987 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73632501-6eb7-4311-88aa-bd5300956afb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 15 14:06:02.671507 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1302/2616] 10.222.0.36 () {58 vars in 1233 bytes} [Sat Oct 15 14:06:02 2022] DELETE /identity/v3/projects/e21a4ed48a484686873dcebe96a0c840 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 15 14:06:12.281695 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:06:12.666442 dv-cinder-861491-3-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 15 14:07:02.635893 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efa6b0-14fd-4730-99ff-b08006469020 service neutron] Authenticating user token {{(pid=87915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 15 14:07:02.637579 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-09efa6b0-14fd-4730-99ff-b08006469020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:07:02.799118 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efa6b0-14fd-4730-99ff-b08006469020 service neutron] Validating token access rules against request {{(pid=87915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 15 14:07:02.862869 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c674d4b-a29f-4ca5-90a1-af8c8c9533c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6720518c03413aa8f85e4d6142fe87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a5a64cc8aa04972bb464407cb0312b4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 15 14:07:02.863488 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c674d4b-a29f-4ca5-90a1-af8c8c9533c3 service placement] REQUEST_METHOD: `GET` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 15 14:07:02.863767 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c674d4b-a29f-4ca5-90a1-af8c8c9533c3 service placement] SCRIPT_NAME: `/identity` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 15 14:07:02.864075 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c674d4b-a29f-4ca5-90a1-af8c8c9533c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 15 14:07:02.865955 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.fernet_utils [None req-2c674d4b-a29f-4ca5-90a1-af8c8c9533c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 15 14:07:02.878023 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c674d4b-a29f-4ca5-90a1-af8c8c9533c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 15 14:07:02.879622 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c674d4b-a29f-4ca5-90a1-af8c8c9533c3 service placement] RBAC: Authorization granted {{(pid=87915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 15 14:07:02.890144 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: [pid: 87915|app: 0|req: 1315/2617] 10.222.0.36 () {60 vars in 1365 bytes} [Sat Oct 15 14:07:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 15 14:07:12.885760 dv-cinder-861491-3-iscsi devstack@keystone.service[87915]: DEBUG dbcounter [-] [87915] Writing DB stats keystone:SELECT=89 {{(pid=87915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}